2024-04-26T00:17:02.776 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T00:17:02.776 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} --archive /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897 --name teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi --owner scheduled_teuthology@teuthology --verbose -- /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897/orig.config.yaml 2024-04-26T00:17:02.807 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T00:17:02.888 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897 branch: squid description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7673897' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: global: mon client directed command retry: 5 mon client hunt interval max multiple: 2 mon election default strategy: 3 mon mgr beacon grace: 90 ms bind msgr2: false ms inject socket failures: 1000 ms type: async mgr: debug mgr: 20 debug monc: 10 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - \(MON_DOWN\) mon_bind_msgr2: false sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 9886 sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 sleep_before_teardown: 0 subset: 30270/100000 suite: rados suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a targets: smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeA4K+d8fVXbH4V0RktXOSYsn8DWHSfnGlr8UEnWBFfDYZ8KMV2Yn/l9COer6ugWFV7wCcZqQmruFMfRQ6FR0g= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - rep_lost_unfound_delete: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_21:00:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T00:17:02.888 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa; will attempt to use it 2024-04-26T00:17:02.889 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks 2024-04-26T00:17:02.889 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T00:17:02.891 INFO:teuthology.task.internal:Checking packages... 2024-04-26T00:17:02.914 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd2c45d977142e024a59b68a8f1dce8f0b9840551' 2024-04-26T00:17:02.915 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T00:17:02.915 INFO:teuthology.packaging:ref: None 2024-04-26T00:17:02.915 INFO:teuthology.packaging:tag: None 2024-04-26T00:17:02.915 INFO:teuthology.packaging:branch: squid 2024-04-26T00:17:02.915 INFO:teuthology.packaging:sha1: d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-26T00:17:02.915 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-26T00:17:03.212 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2447.gb22e2ebd 2024-04-26T00:17:03.213 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T00:17:03.219 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T00:17:03.219 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T00:17:03.228 INFO:teuthology.task.internal:Saving configuration 2024-04-26T00:17:03.240 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T00:17:03.245 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T00:17:03.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 00:11:30.951819', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeA4K+d8fVXbH4V0RktXOSYsn8DWHSfnGlr8UEnWBFfDYZ8KMV2Yn/l9COer6ugWFV7wCcZqQmruFMfRQ6FR0g='} 2024-04-26T00:17:03.271 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T00:17:03.276 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-04-26T00:17:03.276 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T00:17:03.314 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc946672e50>, signals=[15]) 2024-04-26T00:17:03.315 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T00:17:03.320 INFO:teuthology.task.internal:Opening connections... 2024-04-26T00:17:03.320 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:17:03.323 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T00:17:03.400 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T00:17:03.406 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2024-04-26T00:17:03.475 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2024-04-26T00:17:03.476 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-04-26T00:17:03.531 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="9" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="9" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el9" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:LOGO="fedora-logo-icon" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T00:17:03.532 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T00:17:03.533 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-04-26T00:17:03.558 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T00:17:03.566 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T00:17:03.574 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T00:17:03.574 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2024-04-26T00:17:03.590 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T00:17:03.596 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T00:17:03.596 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2024-04-26T00:17:03.654 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T00:17:03.685 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T00:17:03.700 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-26T00:17:03.700 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-26T00:17:03.700 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T00:17:03.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:17:03.716 DEBUG:teuthology.orchestra.run.smithi072:> uname -r 2024-04-26T00:17:03.773 INFO:teuthology.orchestra.run.smithi072.stdout:5.14.0-437.el9.x86_64 2024-04-26T00:17:03.773 INFO:teuthology.task.kernel:Running kernel on smithi072: 5.14.0-437.el9.x86_64 2024-04-26T00:17:03.773 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum install -y kernel 2024-04-26T00:17:06.792 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T00:17:08.905 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 18 MB/s | 19 MB 00:01 2024-04-26T00:17:13.140 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 9.4 MB/s | 6.2 MB 00:00 2024-04-26T00:17:14.597 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-26T00:17:16.996 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 9.2 MB/s | 21 MB 00:02 2024-04-26T00:17:22.069 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 23 kB/s | 1.7 kB 00:00 2024-04-26T00:17:23.968 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T00:17:23.968 INFO:teuthology.orchestra.run.smithi072.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T00:17:24.088 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:17:24.089 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:17:24.089 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:17:24.219 DEBUG:teuthology.orchestra.run.smithi072:> echo no | sudo yum reinstall kernel || true 2024-04-26T00:17:24.758 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 12:17:22 AM UTC. 2024-04-26T00:17:24.889 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:17:24.915 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:17:24.915 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-04-26T00:17:24.915 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:17:24.915 INFO:teuthology.orchestra.run.smithi072.stdout:Reinstalling: 2024-04-26T00:17:24.915 INFO:teuthology.orchestra.run.smithi072.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T00:17:24.916 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:17:24.916 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:17:24.916 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:17:24.916 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:17:24.916 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 255 k 2024-04-26T00:17:24.916 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 0 2024-04-26T00:17:24.916 INFO:teuthology.orchestra.run.smithi072.stderr:Operation aborted. 2024-04-26T00:17:24.987 INFO:teuthology.orchestra.run.smithi072.stdout:Is this ok [y/N]: 2024-04-26T00:17:24.988 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum reinstall -y kernel || true 2024-04-26T00:17:25.483 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 12:17:22 AM UTC. 2024-04-26T00:17:25.598 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:17:25.599 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:17:25.599 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-04-26T00:17:25.599 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:17:25.599 INFO:teuthology.orchestra.run.smithi072.stdout:Reinstalling: 2024-04-26T00:17:25.600 INFO:teuthology.orchestra.run.smithi072.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T00:17:25.600 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:17:25.600 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:17:25.600 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:17:25.600 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:17:25.600 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 255 k 2024-04-26T00:17:25.600 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 0 2024-04-26T00:17:25.600 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:17:25.848 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.0 MB/s | 255 kB 00:00 2024-04-26T00:17:25.849 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:17:25.849 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.0 MB/s | 255 kB 00:00 2024-04-26T00:17:25.868 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:17:26.068 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:17:26.068 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:17:26.073 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:17:26.074 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:17:26.245 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:17:26.337 INFO:teuthology.orchestra.run.smithi072.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T00:17:26.337 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T00:17:27.364 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T00:17:27.364 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T00:17:28.250 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T00:17:28.250 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:17:28.250 INFO:teuthology.orchestra.run.smithi072.stdout:Reinstalled: 2024-04-26T00:17:28.250 INFO:teuthology.orchestra.run.smithi072.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T00:17:28.250 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:17:28.250 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:17:28.459 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T00:17:28.506 INFO:teuthology.orchestra.run.smithi072.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T00:17:28.507 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T00:17:28.507 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T00:17:28.507 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-26T00:17:28.507 DEBUG:teuthology.orchestra.run.smithi072:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T00:17:28.541 INFO:teuthology.orchestra.run.smithi072.stdout:ttyS1 2024-04-26T00:17:28.564 DEBUG:teuthology.parallel:result is None 2024-04-26T00:17:28.565 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T00:17:28.572 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T00:17:28.572 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T00:17:28.621 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T00:17:28.628 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T00:17:28.635 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T00:17:28.635 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T00:17:28.688 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T00:17:28.695 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T00:17:28.695 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T00:17:28.758 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T00:17:28.767 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T00:17:28.769 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T00:17:28.776 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T00:17:28.776 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T00:17:28.832 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T00:17:28.841 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T00:17:28.841 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T00:17:28.887 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T00:17:28.970 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T00:17:29.020 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T00:17:29.101 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T00:17:29.139 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:17:29.140 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T00:17:29.203 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2024-04-26T00:17:29.281 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T00:17:29.747 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T00:17:29.754 INFO:teuthology.task.internal:Starting timer... 2024-04-26T00:17:29.754 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T00:17:29.767 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T00:17:29.775 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-26T00:17:29.776 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2024-04-26T00:17:29.858 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2024-04-26T00:17:29.860 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T00:17:29.861 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2024-04-26T00:17:29.890 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2024-04-26T00:17:29.891 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi072.front.sepia.ceph.com': 'permissive'} 2024-04-26T00:17:29.891 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T00:17:29.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:17:29.955 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T00:17:29.956 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2024-04-26T00:17:30.027 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T00:17:30.038 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T00:17:30.058 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T00:17:30.059 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi072.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T00:20:59.590 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi072.front.sepia.ceph.com')] 2024-04-26T00:20:59.592 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T00:20:59.593 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T00:20:59.673 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-04-26T00:20:59.753 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-04-26T00:20:59.754 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T00:20:59.885 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T00:20:59.885 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T00:20:59.886 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T00:20:59.913 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T00:20:59.926 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T00:20:59.949 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2024-04-26T00:20:59.959 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2024-04-26T00:20:59.972 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T00:20:59.984 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T00:21:00.047 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-04-26T00:21:00.051 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T00:21:00.051 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-04-26T00:21:00.051 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T00:21:00.051 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T00:21:00.051 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T00:21:00.051 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T00:21:00.052 INFO:teuthology.run_tasks:Running task install... 2024-04-26T00:21:00.104 DEBUG:teuthology.task.install:project ceph 2024-04-26T00:21:00.104 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}} 2024-04-26T00:21:00.104 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'} 2024-04-26T00:21:00.104 INFO:teuthology.task.install:Using flavor: default 2024-04-26T00:21:00.111 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T00:21:00.111 INFO:teuthology.task.install:extra packages: [] 2024-04-26T00:21:00.111 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'tag': None, 'wait_for_package': False} 2024-04-26T00:21:00.112 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-26T00:21:00.338 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/ 2024-04-26T00:21:00.339 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2445.gd2c45d97 2024-04-26T00:21:00.450 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/squid/d2c45d977142e024a59b68a8f1dce8f0b9840551/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T00:21:00.450 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:21:00.450 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T00:21:00.475 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T00:21:00.475 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d2c45d977142e024a59b68a8f1dce8f0b9840551/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T00:21:00.539 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T00:21:00.612 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T00:21:00.677 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2024-04-26T00:21:00.678 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2024-04-26T00:21:00.901 INFO:teuthology.orchestra.run.smithi072.stdout:48 files removed 2024-04-26T00:21:00.927 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw 2024-04-26T00:21:01.543 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 232 kB/s | 83 kB 00:00 2024-04-26T00:21:01.910 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-04-26T00:21:02.313 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 5.1 kB/s | 1.9 kB 00:00 2024-04-26T00:21:04.342 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.0 MB 00:01 2024-04-26T00:21:06.570 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 16 MB/s | 19 MB 00:01 2024-04-26T00:21:13.041 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 2.1 MB/s | 6.2 MB 00:02 2024-04-26T00:21:14.631 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 16 kB 00:00 2024-04-26T00:21:34.902 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 21 MB 00:20 2024-04-26T00:21:39.858 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T00:21:42.278 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:21:42.280 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:21:42.280 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:21:42.280 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:21:42.280 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:21:42.280 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 10 M 2024-04-26T00:21:42.280 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-04-26T00:21:42.280 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.4 M 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.4 M 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 21 M 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 25 k 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 745 k 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T00:21:42.281 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 501 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 5.3 M 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 45 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 133 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 162 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 321 k 2024-04-26T00:21:42.282 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 299 k 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 99 k 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:Install 30 Packages 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 2 Packages 2024-04-26T00:21:42.283 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:21:42.284 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 60 M 2024-04-26T00:21:42.284 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:21:43.804 INFO:teuthology.orchestra.run.smithi072.stdout:(1/32): ceph-base-19.0.0-2445.gd2c45d97.el9.x86 4.9 MB/s | 5.4 MB 00:01 2024-04-26T00:21:43.871 INFO:teuthology.orchestra.run.smithi072.stdout:(2/32): ceph-selinux-19.0.0-2445.gd2c45d97.el9. 376 kB/s | 25 kB 00:00 2024-04-26T00:21:44.030 INFO:teuthology.orchestra.run.smithi072.stdout:(3/32): libcephfs2-19.0.0-2445.gd2c45d97.el9.x8 4.6 MB/s | 745 kB 00:00 2024-04-26T00:21:44.188 INFO:teuthology.orchestra.run.smithi072.stdout:(4/32): libradosstriper1-19.0.0-2445.gd2c45d97. 3.1 MB/s | 501 kB 00:00 2024-04-26T00:21:44.672 INFO:teuthology.orchestra.run.smithi072.stdout:(5/32): ceph-radosgw-19.0.0-2445.gd2c45d97.el9. 5.2 MB/s | 10 MB 00:01 2024-04-26T00:21:44.714 INFO:teuthology.orchestra.run.smithi072.stdout:(6/32): python3-ceph-argparse-19.0.0-2445.gd2c4 1.1 MB/s | 45 kB 00:00 2024-04-26T00:21:44.798 INFO:teuthology.orchestra.run.smithi072.stdout:(7/32): python3-ceph-common-19.0.0-2445.gd2c45d 1.6 MB/s | 133 kB 00:00 2024-04-26T00:21:44.874 INFO:teuthology.orchestra.run.smithi072.stdout:(8/32): python3-cephfs-19.0.0-2445.gd2c45d97.el 2.1 MB/s | 162 kB 00:00 2024-04-26T00:21:44.974 INFO:teuthology.orchestra.run.smithi072.stdout:(9/32): python3-rados-19.0.0-2445.gd2c45d97.el9 3.1 MB/s | 321 kB 00:00 2024-04-26T00:21:45.075 INFO:teuthology.orchestra.run.smithi072.stdout:(10/32): python3-rbd-19.0.0-2445.gd2c45d97.el9. 2.9 MB/s | 299 kB 00:00 2024-04-26T00:21:45.234 INFO:teuthology.orchestra.run.smithi072.stdout:(11/32): python3-rgw-19.0.0-2445.gd2c45d97.el9. 627 kB/s | 99 kB 00:00 2024-04-26T00:21:45.553 INFO:teuthology.orchestra.run.smithi072.stdout:(12/32): ceph-common-19.0.0-2445.gd2c45d97.el9. 7.4 MB/s | 21 MB 00:02 2024-04-26T00:21:45.686 INFO:teuthology.orchestra.run.smithi072.stdout:(13/32): librgw2-19.0.0-2445.gd2c45d97.el9.x86_ 3.5 MB/s | 5.3 MB 00:01 2024-04-26T00:21:45.745 INFO:teuthology.orchestra.run.smithi072.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 209 kB/s | 107 kB 00:00 2024-04-26T00:21:45.796 INFO:teuthology.orchestra.run.smithi072.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 420 kB/s | 45 kB 00:00 2024-04-26T00:21:45.846 INFO:teuthology.orchestra.run.smithi072.stdout:(16/32): librdkafka-1.6.1-102.el9.x86_64.rpm 8.7 MB/s | 662 kB 00:00 2024-04-26T00:21:45.880 INFO:teuthology.orchestra.run.smithi072.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-26T00:21:45.906 INFO:teuthology.orchestra.run.smithi072.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 723 kB/s | 42 kB 00:00 2024-04-26T00:21:45.939 INFO:teuthology.orchestra.run.smithi072.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-04-26T00:21:45.973 INFO:teuthology.orchestra.run.smithi072.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 536 kB/s | 36 kB 00:00 2024-04-26T00:21:45.999 INFO:teuthology.orchestra.run.smithi072.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 822 kB/s | 47 kB 00:00 2024-04-26T00:21:46.558 INFO:teuthology.orchestra.run.smithi072.stdout:(22/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 164 kB/s | 160 kB 00:00 2024-04-26T00:21:46.858 INFO:teuthology.orchestra.run.smithi072.stdout:(23/32): libarrow-doc-9.0.0-10.el9.noarch.rpm 86 kB/s | 26 kB 00:00 2024-04-26T00:21:46.934 INFO:teuthology.orchestra.run.smithi072.stdout:(24/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 322 kB/s | 309 kB 00:00 2024-04-26T00:21:47.067 INFO:teuthology.orchestra.run.smithi072.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 232 kB/s | 49 kB 00:00 2024-04-26T00:21:47.135 INFO:teuthology.orchestra.run.smithi072.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 336 kB/s | 67 kB 00:00 2024-04-26T00:21:47.527 INFO:teuthology.orchestra.run.smithi072.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 489 kB/s | 192 kB 00:00 2024-04-26T00:21:48.922 INFO:teuthology.orchestra.run.smithi072.stdout:(28/32): parquet-libs-9.0.0-10.el9.x86_64.rpm 452 kB/s | 838 kB 00:01 2024-04-26T00:21:49.281 INFO:teuthology.orchestra.run.smithi072.stdout:(29/32): librados2-19.0.0-2445.gd2c45d97.el9.x8 9.4 MB/s | 3.4 MB 00:00 2024-04-26T00:21:49.539 INFO:teuthology.orchestra.run.smithi072.stdout:(30/32): librbd1-19.0.0-2445.gd2c45d97.el9.x86_ 12 MB/s | 3.1 MB 00:00 2024-04-26T00:21:50.156 INFO:teuthology.orchestra.run.smithi072.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 616 kB/s | 1.6 MB 00:02 2024-04-26T00:21:53.323 INFO:teuthology.orchestra.run.smithi072.stdout:(32/32): libarrow-9.0.0-10.el9.x86_64.rpm 618 kB/s | 4.4 MB 00:07 2024-04-26T00:21:53.328 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:21:53.328 INFO:teuthology.orchestra.run.smithi072.stdout:Total 5.4 MB/s | 60 MB 00:11 2024-04-26T00:21:53.424 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:21:53.461 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:21:53.462 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:21:53.929 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:21:53.929 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:21:55.465 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:21:55.550 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-26T00:21:55.589 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-26T00:21:55.624 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-26T00:21:55.656 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T00:21:55.675 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-26T00:21:55.874 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T00:21:55.929 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-26T00:21:56.312 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-26T00:21:56.441 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/34 2024-04-26T00:21:56.534 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/34 2024-04-26T00:21:56.624 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 8/34 2024-04-26T00:21:56.652 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/34 2024-04-26T00:21:56.715 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 9/34 2024-04-26T00:21:56.753 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-26T00:21:56.797 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9. 11/34 2024-04-26T00:21:56.867 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/34 2024-04-26T00:21:56.891 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_6 13/34 2024-04-26T00:21:56.955 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_6 13/34 2024-04-26T00:21:56.990 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-26T00:21:57.101 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-26T00:21:57.133 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 16/34 2024-04-26T00:21:57.174 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-26T00:21:57.209 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-26T00:21:57.246 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 19/34 2024-04-26T00:21:57.281 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-26T00:21:57.599 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-26T00:21:57.654 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 22/34 2024-04-26T00:21:57.744 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 22/34 2024-04-26T00:21:57.775 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 23/34 2024-04-26T00:21:57.812 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-26T00:21:58.284 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 25/34 2024-04-26T00:21:58.316 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/34 2024-04-26T00:21:58.389 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 26/34 2024-04-26T00:21:58.467 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 27/34 2024-04-26T00:21:58.505 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x8 28/34 2024-04-26T00:22:00.764 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/34 2024-04-26T00:22:00.800 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/34 2024-04-26T00:22:01.341 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 29/34 2024-04-26T00:22:01.383 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 30/34 2024-04-26T00:22:01.480 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 30/34 2024-04-26T00:22:01.480 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T00:22:01.480 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T00:22:01.480 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:01.510 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 31/34 2024-04-26T00:22:12.657 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 31/34 2024-04-26T00:22:12.657 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2024-04-26T00:22:12.657 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2024-04-26T00:22:12.658 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2024-04-26T00:22:12.658 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2024-04-26T00:22:12.658 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2024-04-26T00:22:12.658 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /root 2024-04-26T00:22:12.658 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2024-04-26T00:22:12.658 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:13.625 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 32/34 2024-04-26T00:22:13.649 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 32/34 2024-04-26T00:22:13.649 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T00:22:13.649 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T00:22:13.649 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T00:22:13.649 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T00:22:13.650 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:13.651 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T00:22:13.746 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-26T00:22:13.746 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T00:22:15.203 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T00:22:15.203 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/34 2024-04-26T00:22:15.203 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/34 2024-04-26T00:22:15.203 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/34 2024-04-26T00:22:15.203 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/34 2024-04-26T00:22:15.203 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 5/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_6 6/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 7/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9. 8/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x8 9/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 10/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 11/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 12/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 13/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-26T00:22:15.204 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 24/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 25/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 28/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 31/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-26T00:22:15.205 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 33/34 2024-04-26T00:22:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-26T00:22:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:16.032 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-04-26T00:22:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.032 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.032 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T00:22:16.033 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T00:22:16.034 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T00:22:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T00:22:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T00:22:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T00:22:16.035 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:16.035 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:22:16.347 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-test 2024-04-26T00:22:16.953 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:37 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:22:17.082 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:22:17.082 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:22:17.082 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:17.082 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:22:17.082 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 48 M 2024-04-26T00:22:17.082 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:22:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 163 k 2024-04-26T00:22:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T00:22:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T00:22:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:17.083 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:22:17.083 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:17.083 INFO:teuthology.orchestra.run.smithi072.stdout:Install 4 Packages 2024-04-26T00:22:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:17.084 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 49 M 2024-04-26T00:22:17.084 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 207 M 2024-04-26T00:22:17.084 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:22:17.588 INFO:teuthology.orchestra.run.smithi072.stdout:(1/4): libcephsqlite-19.0.0-2445.gd2c45d97.el9. 434 kB/s | 163 kB 00:00 2024-04-26T00:22:18.225 INFO:teuthology.orchestra.run.smithi072.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 301 kB/s | 305 kB 00:01 2024-04-26T00:22:18.267 INFO:teuthology.orchestra.run.smithi072.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 94 kB/s | 64 kB 00:00 2024-04-26T00:22:20.263 INFO:teuthology.orchestra.run.smithi072.stdout:(4/4): ceph-test-19.0.0-2445.gd2c45d97.el9.x86_ 16 MB/s | 48 MB 00:03 2024-04-26T00:22:20.266 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:22:20.266 INFO:teuthology.orchestra.run.smithi072.stdout:Total 15 MB/s | 49 MB 00:03 2024-04-26T00:22:20.293 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:22:20.314 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:22:20.314 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:22:20.677 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:22:20.677 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:22:21.186 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:22:21.272 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T00:22:21.314 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T00:22:21.343 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/4 2024-04-26T00:22:25.843 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/4 2024-04-26T00:22:25.929 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/4 2024-04-26T00:22:29.687 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/4 2024-04-26T00:22:29.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/4 2024-04-26T00:22:29.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/4 2024-04-26T00:22:29.687 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T00:22:29.940 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T00:22:29.940 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:29.940 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:22:29.940 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:29.940 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:29.940 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T00:22:29.940 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T00:22:29.941 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:29.941 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:22:30.254 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph 2024-04-26T00:22:30.752 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:00:51 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:22:30.872 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:22:30.874 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 6.6 k 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 2.4 M 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 1.5 M 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 249 k 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 4.7 M 2024-04-26T00:22:30.875 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 17 M 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T00:22:30.876 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T00:22:30.877 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T00:22:30.878 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:30.879 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:22:30.879 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:30.879 INFO:teuthology.orchestra.run.smithi072.stdout:Install 36 Packages 2024-04-26T00:22:30.879 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:30.880 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 30 M 2024-04-26T00:22:30.880 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 107 M 2024-04-26T00:22:30.880 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:22:33.783 INFO:teuthology.orchestra.run.smithi072.stdout:(1/36): ceph-19.0.0-2445.gd2c45d97.el9.x86_64.r 6.1 kB/s | 6.6 kB 00:01 2024-04-26T00:22:34.692 INFO:teuthology.orchestra.run.smithi072.stdout:(2/36): ceph-mon-19.0.0-2445.gd2c45d97.el9.x86_ 5.2 MB/s | 4.7 MB 00:00 2024-04-26T00:22:34.743 INFO:teuthology.orchestra.run.smithi072.stdout:(3/36): ceph-mgr-19.0.0-2445.gd2c45d97.el9.x86_ 749 kB/s | 1.5 MB 00:02 2024-04-26T00:22:34.910 INFO:teuthology.orchestra.run.smithi072.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2445.gd2c4 1.5 MB/s | 249 kB 00:00 2024-04-26T00:22:35.235 INFO:teuthology.orchestra.run.smithi072.stdout:(5/36): ceph-mds-19.0.0-2445.gd2c45d97.el9.x86_ 955 kB/s | 2.4 MB 00:02 2024-04-26T00:22:35.269 INFO:teuthology.orchestra.run.smithi072.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 705 kB/s | 253 kB 00:00 2024-04-26T00:22:35.637 INFO:teuthology.orchestra.run.smithi072.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 3.1 MB/s | 1.2 MB 00:00 2024-04-26T00:22:35.662 INFO:teuthology.orchestra.run.smithi072.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 271 kB/s | 106 kB 00:00 2024-04-26T00:22:35.697 INFO:teuthology.orchestra.run.smithi072.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.2 MB/s | 135 kB 00:00 2024-04-26T00:22:35.731 INFO:teuthology.orchestra.run.smithi072.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 1.8 MB/s | 125 kB 00:00 2024-04-26T00:22:35.782 INFO:teuthology.orchestra.run.smithi072.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-04-26T00:22:35.807 INFO:teuthology.orchestra.run.smithi072.stdout:(12/36): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 188 kB 00:00 2024-04-26T00:22:35.858 INFO:teuthology.orchestra.run.smithi072.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-26T00:22:35.883 INFO:teuthology.orchestra.run.smithi072.stdout:(14/36): python3-markupsafe-1.1.1-12.el9.x86_64 459 kB/s | 35 kB 00:00 2024-04-26T00:22:35.909 INFO:teuthology.orchestra.run.smithi072.stdout:(15/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-04-26T00:22:35.934 INFO:teuthology.orchestra.run.smithi072.stdout:(16/36): python3-toml-0.10.2-6.el9.noarch.rpm 826 kB/s | 42 kB 00:00 2024-04-26T00:22:35.960 INFO:teuthology.orchestra.run.smithi072.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 441 kB/s | 22 kB 00:00 2024-04-26T00:22:36.238 INFO:teuthology.orchestra.run.smithi072.stdout:(18/36): ceph-osd-19.0.0-2445.gd2c45d97.el9.x86 11 MB/s | 17 MB 00:01 2024-04-26T00:22:36.411 INFO:teuthology.orchestra.run.smithi072.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 96 kB/s | 43 kB 00:00 2024-04-26T00:22:36.495 INFO:teuthology.orchestra.run.smithi072.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 270 kB/s | 151 kB 00:00 2024-04-26T00:22:36.553 INFO:teuthology.orchestra.run.smithi072.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 541 kB/s | 172 kB 00:00 2024-04-26T00:22:36.579 INFO:teuthology.orchestra.run.smithi072.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 127 kB/s | 11 kB 00:00 2024-04-26T00:22:36.629 INFO:teuthology.orchestra.run.smithi072.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 234 kB/s | 18 kB 00:00 2024-04-26T00:22:36.657 INFO:teuthology.orchestra.run.smithi072.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-04-26T00:22:36.682 INFO:teuthology.orchestra.run.smithi072.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 225 kB/s | 23 kB 00:00 2024-04-26T00:22:36.716 INFO:teuthology.orchestra.run.smithi072.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 226 kB/s | 19 kB 00:00 2024-04-26T00:22:36.741 INFO:teuthology.orchestra.run.smithi072.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 232 kB/s | 20 kB 00:00 2024-04-26T00:22:36.767 INFO:teuthology.orchestra.run.smithi072.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 547 kB/s | 46 kB 00:00 2024-04-26T00:22:36.802 INFO:teuthology.orchestra.run.smithi072.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 933 kB/s | 79 kB 00:00 2024-04-26T00:22:36.935 INFO:teuthology.orchestra.run.smithi072.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 1.4 MB/s | 272 kB 00:00 2024-04-26T00:22:36.960 INFO:teuthology.orchestra.run.smithi072.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 85 kB/s | 16 kB 00:00 2024-04-26T00:22:36.986 INFO:teuthology.orchestra.run.smithi072.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 487 kB/s | 90 kB 00:00 2024-04-26T00:22:37.020 INFO:teuthology.orchestra.run.smithi072.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 424 kB/s | 36 kB 00:00 2024-04-26T00:22:37.095 INFO:teuthology.orchestra.run.smithi072.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-04-26T00:22:37.220 INFO:teuthology.orchestra.run.smithi072.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.8 MB/s | 427 kB 00:00 2024-04-26T00:22:37.254 INFO:teuthology.orchestra.run.smithi072.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 85 kB/s | 20 kB 00:00 2024-04-26T00:22:37.258 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:22:37.261 INFO:teuthology.orchestra.run.smithi072.stdout:Total 4.7 MB/s | 30 MB 00:06 2024-04-26T00:22:37.397 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:22:37.437 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:22:37.437 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:22:37.755 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:22:37.755 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:22:38.994 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:22:39.058 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-26T00:22:39.101 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-26T00:22:39.138 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-26T00:22:39.217 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-26T00:22:39.263 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-26T00:22:39.296 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-26T00:22:39.388 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-26T00:22:39.477 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-26T00:22:39.527 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-26T00:22:39.572 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-26T00:22:39.610 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-26T00:22:39.646 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-26T00:22:39.679 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-26T00:22:39.712 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-26T00:22:39.758 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-26T00:22:39.793 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-26T00:22:39.831 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-26T00:22:39.905 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-26T00:22:40.053 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-26T00:22:40.144 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-26T00:22:40.198 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-26T00:22:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-26T00:22:40.303 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-26T00:22:40.368 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-26T00:22:40.565 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-26T00:22:40.621 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-26T00:22:40.699 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-26T00:22:40.790 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-26T00:22:40.830 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-26T00:22:40.948 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-26T00:22:41.115 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9. 31/36 2024-04-26T00:22:41.145 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 32/36 2024-04-26T00:22:41.171 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 32/36 2024-04-26T00:22:41.171 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T00:22:41.171 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T00:22:41.171 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T00:22:41.171 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T00:22:41.171 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:42.605 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 33/36 2024-04-26T00:22:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 33/36 2024-04-26T00:22:42.632 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T00:22:42.632 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T00:22:42.632 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T00:22:42.632 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T00:22:42.632 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:43.075 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 34/36 2024-04-26T00:22:43.098 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 34/36 2024-04-26T00:22:43.098 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T00:22:43.099 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T00:22:43.099 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T00:22:43.099 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T00:22:43.099 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:43.318 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 35/36 2024-04-26T00:22:43.341 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 35/36 2024-04-26T00:22:43.341 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T00:22:43.341 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T00:22:43.341 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T00:22:43.341 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T00:22:43.341 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:43.406 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 36/36 2024-04-26T00:22:45.955 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 36/36 2024-04-26T00:22:45.955 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/36 2024-04-26T00:22:45.955 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/36 2024-04-26T00:22:45.955 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 3/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 4/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 5/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9. 6/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-26T00:22:45.956 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-26T00:22:45.957 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-26T00:22:45.958 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-26T00:22:45.958 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-26T00:22:45.959 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-26T00:22:45.959 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-26T00:22:45.959 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-26T00:22:45.959 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-26T00:22:45.959 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-26T00:22:45.959 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-26T00:22:46.791 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-26T00:22:46.791 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:46.791 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:22:46.791 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T00:22:46.792 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T00:22:46.793 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T00:22:46.794 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T00:22:46.794 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T00:22:46.794 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T00:22:46.794 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T00:22:46.794 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T00:22:46.794 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:46.794 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:22:47.174 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-base 2024-04-26T00:22:47.690 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:08 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:22:47.768 INFO:teuthology.orchestra.run.smithi072.stdout:Package ceph-base-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:22:47.809 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:22:47.810 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:22:47.810 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:22:47.889 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install cephadm 2024-04-26T00:22:48.393 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:09 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:22:48.512 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:22:48.512 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:48.512 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:22:48.512 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:48.513 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:22:48.513 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 762 k 2024-04-26T00:22:48.513 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:48.513 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:22:48.513 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:48.513 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-04-26T00:22:48.513 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:48.514 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 762 k 2024-04-26T00:22:48.514 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 768 k 2024-04-26T00:22:48.514 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:22:48.840 INFO:teuthology.orchestra.run.smithi072.stdout:cephadm-19.0.0-2445.gd2c45d97.el9.noarch.rpm 2.3 MB/s | 762 kB 00:00 2024-04-26T00:22:48.840 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:22:48.840 INFO:teuthology.orchestra.run.smithi072.stdout:Total 2.3 MB/s | 762 kB 00:00 2024-04-26T00:22:48.840 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:22:48.845 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:22:48.845 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:22:48.851 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:22:48.851 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:22:48.921 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:22:49.407 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 1/1 2024-04-26T00:22:49.489 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 1/1 2024-04-26T00:22:49.820 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 1/1 2024-04-26T00:22:50.062 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 1/1 2024-04-26T00:22:50.062 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:50.062 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:22:50.062 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:22:50.063 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:50.063 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:22:50.301 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T00:22:50.819 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:11 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:22:50.940 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 145 k 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-04-26T00:22:50.941 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:50.942 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 145 k 2024-04-26T00:22:50.942 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 435 k 2024-04-26T00:22:50.942 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:22:51.170 INFO:teuthology.orchestra.run.smithi072.stdout:ceph-immutable-object-cache-19.0.0-2445.gd2c45d 634 kB/s | 145 kB 00:00 2024-04-26T00:22:51.171 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:22:51.171 INFO:teuthology.orchestra.run.smithi072.stdout:Total 629 kB/s | 145 kB 00:00 2024-04-26T00:22:51.172 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:22:51.179 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:22:51.179 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:22:51.234 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:22:51.235 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:22:51.382 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:22:51.473 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97. 1/1 2024-04-26T00:22:51.497 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97. 1/1 2024-04-26T00:22:51.497 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T00:22:51.497 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T00:22:51.497 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:52.364 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97. 1/1 2024-04-26T00:22:52.364 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:52.365 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:22:52.365 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:22:52.365 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:52.365 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:22:52.643 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr 2024-04-26T00:22:53.155 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:14 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:22:53.234 INFO:teuthology.orchestra.run.smithi072.stdout:Package ceph-mgr-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:22:53.276 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:22:53.277 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:22:53.277 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:22:53.378 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T00:22:53.890 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:14 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:22:54.010 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:22:54.011 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:54.011 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:22:54.011 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:54.011 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:22:54.011 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 3.5 M 2024-04-26T00:22:54.011 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:22:54.011 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T00:22:54.011 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 32 k 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 15 k 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout:Install 12 Packages 2024-04-26T00:22:54.012 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:22:54.013 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 8.6 M 2024-04-26T00:22:54.013 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 103 M 2024-04-26T00:22:54.013 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:22:55.173 INFO:teuthology.orchestra.run.smithi072.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2445.gd2 88 kB/s | 32 kB 00:00 2024-04-26T00:22:55.232 INFO:teuthology.orchestra.run.smithi072.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2445.gd2c 36 kB/s | 15 kB 00:00 2024-04-26T00:22:55.358 INFO:teuthology.orchestra.run.smithi072.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-04-26T00:22:55.458 INFO:teuthology.orchestra.run.smithi072.stdout:(4/12): protobuf-compiler-3.14.0-13.el9.x86_64. 11 MB/s | 863 kB 00:00 2024-04-26T00:22:55.917 INFO:teuthology.orchestra.run.smithi072.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2445.gd2c45d9 3.2 MB/s | 3.5 MB 00:01 2024-04-26T00:22:56.376 INFO:teuthology.orchestra.run.smithi072.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-04-26T00:22:56.551 INFO:teuthology.orchestra.run.smithi072.stdout:(7/12): protobuf-3.14.0-13.el9.x86_64.rpm 750 kB/s | 1.0 MB 00:01 2024-04-26T00:22:56.677 INFO:teuthology.orchestra.run.smithi072.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 450 kB/s | 548 kB 00:01 2024-04-26T00:22:56.810 INFO:teuthology.orchestra.run.smithi072.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 285 kB/s | 31 kB 00:00 2024-04-26T00:22:57.078 INFO:teuthology.orchestra.run.smithi072.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 710 kB/s | 188 kB 00:00 2024-04-26T00:22:57.311 INFO:teuthology.orchestra.run.smithi072.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 190 kB/s | 144 kB 00:00 2024-04-26T00:22:58.912 INFO:teuthology.orchestra.run.smithi072.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 823 kB/s | 2.0 MB 00:02 2024-04-26T00:22:58.920 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:22:58.920 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.8 MB/s | 8.6 MB 00:04 2024-04-26T00:22:58.970 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:22:58.988 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:22:58.988 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:22:59.142 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:22:59.143 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:22:59.669 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:22:59.738 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-26T00:22:59.784 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-26T00:22:59.840 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-26T00:22:59.878 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-26T00:22:59.956 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-26T00:23:00.093 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-26T00:23:00.188 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-26T00:23:00.232 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-26T00:23:00.269 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-26T00:23:00.311 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9 10/12 2024-04-26T00:23:01.353 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el 11/12 2024-04-26T00:23:01.397 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noa 12/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noa 12/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el 1/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noa 2/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9 3/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-26T00:23:03.628 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-26T00:23:03.629 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T00:23:03.902 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T00:23:03.903 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T00:23:03.903 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T00:23:03.903 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T00:23:03.903 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T00:23:03.903 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T00:23:03.903 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T00:23:03.903 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:03.903 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:04.178 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T00:23:04.686 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:25 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:04.808 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 7.4 M 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T00:23:04.809 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout:Install 12 Packages 2024-04-26T00:23:04.810 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:04.811 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 38 M 2024-04-26T00:23:04.811 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 209 M 2024-04-26T00:23:04.811 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:05.276 INFO:teuthology.orchestra.run.smithi072.stdout:(1/12): libgfortran-11.4.1-3.el9.x86_64.rpm 4.0 MB/s | 803 kB 00:00 2024-04-26T00:23:05.343 INFO:teuthology.orchestra.run.smithi072.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 768 kB/s | 33 kB 00:00 2024-04-26T00:23:05.527 INFO:teuthology.orchestra.run.smithi072.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 16 MB/s | 3.0 MB 00:00 2024-04-26T00:23:05.571 INFO:teuthology.orchestra.run.smithi072.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 432 kB/s | 18 kB 00:00 2024-04-26T00:23:05.605 INFO:teuthology.orchestra.run.smithi072.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 1.0 MB/s | 35 kB 00:00 2024-04-26T00:23:05.805 INFO:teuthology.orchestra.run.smithi072.stdout:(6/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 24 MB/s | 4.7 MB 00:00 2024-04-26T00:23:05.856 INFO:teuthology.orchestra.run.smithi072.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-26T00:23:06.073 INFO:teuthology.orchestra.run.smithi072.stdout:(8/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 24 MB/s | 5.1 MB 00:00 2024-04-26T00:23:06.150 INFO:teuthology.orchestra.run.smithi072.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 3.3 MB/s | 250 kB 00:00 2024-04-26T00:23:06.199 INFO:teuthology.orchestra.run.smithi072.stdout:(10/12): libquadmath-11.4.1-3.el9.x86_64.rpm 174 kB/s | 195 kB 00:01 2024-04-26T00:23:06.833 INFO:teuthology.orchestra.run.smithi072.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-26T00:23:07.808 INFO:teuthology.orchestra.run.smithi072.stdout:(12/12): ceph-mgr-diskprediction-local-19.0.0-2 2.7 MB/s | 7.4 MB 00:02 2024-04-26T00:23:07.812 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:07.812 INFO:teuthology.orchestra.run.smithi072.stdout:Total 13 MB/s | 38 MB 00:03 2024-04-26T00:23:08.164 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:08.191 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:08.191 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:08.598 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:08.598 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:09.431 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:09.497 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T00:23:09.541 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T00:23:09.642 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T00:23:09.687 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T00:23:09.866 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T00:23:09.895 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T00:23:09.972 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T00:23:10.622 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T00:23:10.751 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T00:23:11.992 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T00:23:13.058 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T00:23:13.079 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45 12/12 2024-04-26T00:23:17.083 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45 12/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45 1/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T00:23:17.084 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T00:23:17.447 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T00:23:17.447 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:17.447 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:17.447 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:23:17.447 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T00:23:17.447 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T00:23:17.447 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:17.448 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:17.921 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-rook 2024-04-26T00:23:18.441 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:39 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:18.565 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:18.566 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:18.566 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:23:18.566 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:18.566 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 49 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T00:23:18.567 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T00:23:18.568 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T00:23:18.568 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T00:23:18.568 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:18.568 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:18.568 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:18.568 INFO:teuthology.orchestra.run.smithi072.stdout:Install 13 Packages 2024-04-26T00:23:18.568 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:18.569 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 2.2 M 2024-04-26T00:23:18.569 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 26 M 2024-04-26T00:23:18.569 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:19.168 INFO:teuthology.orchestra.run.smithi072.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 187 kB/s | 19 kB 00:00 2024-04-26T00:23:19.285 INFO:teuthology.orchestra.run.smithi072.stdout:(2/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.3 MB/s | 222 kB 00:00 2024-04-26T00:23:19.328 INFO:teuthology.orchestra.run.smithi072.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.8 MB/s | 159 kB 00:00 2024-04-26T00:23:19.378 INFO:teuthology.orchestra.run.smithi072.stdout:(4/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.4 MB/s | 279 kB 00:00 2024-04-26T00:23:19.579 INFO:teuthology.orchestra.run.smithi072.stdout:(5/13): python3-jsonpatch-1.21-16.el9.noarch.rp 50 kB/s | 26 kB 00:00 2024-04-26T00:23:19.630 INFO:teuthology.orchestra.run.smithi072.stdout:(6/13): ceph-mgr-rook-19.0.0-2445.gd2c45d97.el9 87 kB/s | 49 kB 00:00 2024-04-26T00:23:19.738 INFO:teuthology.orchestra.run.smithi072.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 202 kB/s | 32 kB 00:00 2024-04-26T00:23:19.764 INFO:teuthology.orchestra.run.smithi072.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 112 kB/s | 15 kB 00:00 2024-04-26T00:23:19.822 INFO:teuthology.orchestra.run.smithi072.stdout:(9/13): python3-requests-oauthlib-1.3.0-12.el9. 121 kB/s | 54 kB 00:00 2024-04-26T00:23:19.931 INFO:teuthology.orchestra.run.smithi072.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 1.0 MB/s | 206 kB 00:00 2024-04-26T00:23:19.990 INFO:teuthology.orchestra.run.smithi072.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 353 kB/s | 59 kB 00:00 2024-04-26T00:23:20.015 INFO:teuthology.orchestra.run.smithi072.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.0 MB/s | 90 kB 00:00 2024-04-26T00:23:20.058 INFO:teuthology.orchestra.run.smithi072.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 3.5 MB/s | 1.0 MB 00:00 2024-04-26T00:23:20.060 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:20.060 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.5 MB/s | 2.2 MB 00:01 2024-04-26T00:23:20.129 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:20.145 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:20.145 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:20.229 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:20.229 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:20.730 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:20.898 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T00:23:20.962 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T00:23:21.026 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T00:23:21.065 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T00:23:21.106 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T00:23:21.200 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T00:23:21.313 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T00:23:21.371 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T00:23:22.094 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T00:23:22.157 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T00:23:22.191 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T00:23:22.240 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T00:23:22.270 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 13/13 2024-04-26T00:23:23.257 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 13/13 2024-04-26T00:23:23.257 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 1/13 2024-04-26T00:23:23.257 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T00:23:23.258 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T00:23:23.558 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T00:23:23.559 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T00:23:23.559 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T00:23:23.559 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T00:23:23.559 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T00:23:23.559 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T00:23:23.559 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T00:23:23.559 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:23.559 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:23.865 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T00:23:24.377 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:45 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:24.495 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:24.496 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:24.496 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:23:24.496 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:24.496 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:24.496 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 141 k 2024-04-26T00:23:24.496 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout:Install 6 Packages 2024-04-26T00:23:24.497 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:24.498 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 7.0 M 2024-04-26T00:23:24.498 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 33 M 2024-04-26T00:23:24.498 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:25.094 INFO:teuthology.orchestra.run.smithi072.stdout:(1/6): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-04-26T00:23:25.194 INFO:teuthology.orchestra.run.smithi072.stdout:(2/6): ceph-mgr-cephadm-19.0.0-2445.gd2c45d97.e 330 kB/s | 141 kB 00:00 2024-04-26T00:23:25.353 INFO:teuthology.orchestra.run.smithi072.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 363 kB/s | 58 kB 00:00 2024-04-26T00:23:25.387 INFO:teuthology.orchestra.run.smithi072.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.8 MB/s | 548 kB 00:00 2024-04-26T00:23:25.412 INFO:teuthology.orchestra.run.smithi072.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 871 kB/s | 51 kB 00:00 2024-04-26T00:23:25.938 INFO:teuthology.orchestra.run.smithi072.stdout:(6/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 212 kB/s | 248 kB 00:01 2024-04-26T00:23:25.941 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:25.942 INFO:teuthology.orchestra.run.smithi072.stdout:Total 4.8 MB/s | 7.0 MB 00:01 2024-04-26T00:23:26.013 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:26.025 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:26.025 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:26.121 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:26.121 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:26.418 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:26.577 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T00:23:26.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T00:23:27.159 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T00:23:27.247 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T00:23:27.311 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T00:23:27.333 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 6/6 2024-04-26T00:23:28.388 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 6/6 2024-04-26T00:23:28.388 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 1/6 2024-04-26T00:23:28.388 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T00:23:28.388 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T00:23:28.388 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T00:23:28.388 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T00:23:28.667 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T00:23:28.667 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:28.667 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:28.668 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:23:28.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T00:23:28.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T00:23:28.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T00:23:28.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T00:23:28.668 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T00:23:28.668 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:28.668 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:28.947 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-fuse 2024-04-26T00:23:29.458 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:50 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:29.579 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 890 k 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout:Install 2 Packages 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:29.580 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 970 k 2024-04-26T00:23:29.581 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 2.8 M 2024-04-26T00:23:29.581 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:30.044 INFO:teuthology.orchestra.run.smithi072.stdout:(1/2): ceph-fuse-19.0.0-2445.gd2c45d97.el9.x86_ 2.7 MB/s | 890 kB 00:00 2024-04-26T00:23:30.536 INFO:teuthology.orchestra.run.smithi072.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 99 kB/s | 80 kB 00:00 2024-04-26T00:23:30.537 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:30.537 INFO:teuthology.orchestra.run.smithi072.stdout:Total 1.0 MB/s | 970 kB 00:00 2024-04-26T00:23:30.549 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:30.572 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:30.572 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:30.652 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:30.653 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:30.835 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:30.982 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T00:23:31.053 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/2 2024-04-26T00:23:31.732 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2/2 2024-04-26T00:23:31.732 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/2 2024-04-26T00:23:31.979 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T00:23:31.979 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:31.979 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:31.979 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T00:23:31.980 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:31.980 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:32.272 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-volume 2024-04-26T00:23:32.789 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:53 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:32.910 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:32.911 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:32.911 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-04-26T00:23:32.911 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:32.911 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:32.911 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:19.0.0-2445.gd2c45d97.el9 ceph-noarch 264 k 2024-04-26T00:23:32.911 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-04-26T00:23:32.911 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T00:23:32.911 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:32.912 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:32.912 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:32.912 INFO:teuthology.orchestra.run.smithi072.stdout:Install 2 Packages 2024-04-26T00:23:32.912 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:32.912 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 341 k 2024-04-26T00:23:32.912 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 1.5 M 2024-04-26T00:23:32.912 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:33.459 INFO:teuthology.orchestra.run.smithi072.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 304 kB/s | 77 kB 00:00 2024-04-26T00:23:33.809 INFO:teuthology.orchestra.run.smithi072.stdout:(2/2): ceph-volume-19.0.0-2445.gd2c45d97.el9.no 436 kB/s | 264 kB 00:00 2024-04-26T00:23:33.811 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:33.811 INFO:teuthology.orchestra.run.smithi072.stdout:Total 379 kB/s | 341 kB 00:00 2024-04-26T00:23:33.823 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:33.833 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:33.833 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:33.859 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:33.859 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:33.999 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:34.189 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-26T00:23:34.232 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2/2 2024-04-26T00:23:34.248 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2/2 2024-04-26T00:23:34.248 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T00:23:34.248 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T00:23:34.248 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:35.103 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 1/2 2024-04-26T00:23:35.379 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-26T00:23:35.379 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:35.379 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:35.379 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:19.0.0-2445.gd2c45d97.el9.noarch 2024-04-26T00:23:35.379 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T00:23:35.379 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:35.379 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:35.778 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librados-devel 2024-04-26T00:23:36.305 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:57 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:36.429 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:36.429 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 127 k 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 127 k 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 456 k 2024-04-26T00:23:36.430 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:36.667 INFO:teuthology.orchestra.run.smithi072.stdout:librados-devel-19.0.0-2445.gd2c45d97.el9.x86_64 536 kB/s | 127 kB 00:00 2024-04-26T00:23:36.667 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:36.667 INFO:teuthology.orchestra.run.smithi072.stdout:Total 533 kB/s | 127 kB 00:00 2024-04-26T00:23:36.668 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:36.673 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:36.673 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:36.736 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:36.736 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:36.883 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:36.989 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:37.435 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:37.667 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:37.667 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:37.667 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:37.667 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:23:37.667 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:37.667 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:37.933 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install libcephfs2 2024-04-26T00:23:38.449 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:01:59 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:38.528 INFO:teuthology.orchestra.run.smithi072.stdout:Package libcephfs2-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:23:38.569 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:38.570 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:23:38.570 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:38.665 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install libcephfs-devel 2024-04-26T00:23:39.181 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:00 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 32 k 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 32 k 2024-04-26T00:23:39.302 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 142 k 2024-04-26T00:23:39.303 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:39.470 INFO:teuthology.orchestra.run.smithi072.stdout:libcephfs-devel-19.0.0-2445.gd2c45d97.el9.x86_6 189 kB/s | 32 kB 00:00 2024-04-26T00:23:39.471 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:39.471 INFO:teuthology.orchestra.run.smithi072.stdout:Total 188 kB/s | 32 kB 00:00 2024-04-26T00:23:39.471 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:39.474 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:39.474 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:39.476 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:39.476 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:39.559 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:39.673 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:40.025 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:40.213 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:40.213 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:40.213 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:40.214 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:23:40.214 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:40.214 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:40.411 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librados2 2024-04-26T00:23:40.921 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:01 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:41.000 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:23:41.040 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:41.041 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:23:41.041 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:41.119 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install librbd1 2024-04-26T00:23:41.628 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:02 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:41.707 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:23:41.749 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:41.750 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:23:41.750 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:41.822 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rados 2024-04-26T00:23:42.331 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:03 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:42.410 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rados-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:23:42.450 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:42.451 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:23:42.451 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:42.524 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rgw 2024-04-26T00:23:43.033 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:04 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:43.112 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rgw-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:23:43.153 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:43.154 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:23:43.154 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:43.224 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-cephfs 2024-04-26T00:23:43.735 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:04 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:43.818 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-cephfs-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:23:43.856 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:43.857 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:23:43.857 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:43.933 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install python3-rbd 2024-04-26T00:23:44.442 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:05 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:44.521 INFO:teuthology.orchestra.run.smithi072.stdout:Package python3-rbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 is already installed. 2024-04-26T00:23:44.561 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:44.562 INFO:teuthology.orchestra.run.smithi072.stdout:Nothing to do. 2024-04-26T00:23:44.563 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:44.641 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-fuse 2024-04-26T00:23:45.155 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:06 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:45.281 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 85 k 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:45.282 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-04-26T00:23:45.283 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:45.283 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 85 k 2024-04-26T00:23:45.283 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 231 k 2024-04-26T00:23:45.283 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:45.578 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-fuse-19.0.0-2445.gd2c45d97.el9.x86_64.rpm 289 kB/s | 85 kB 00:00 2024-04-26T00:23:45.578 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:45.579 INFO:teuthology.orchestra.run.smithi072.stdout:Total 288 kB/s | 85 kB 00:00 2024-04-26T00:23:45.579 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:45.585 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:45.586 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:45.649 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:45.649 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:45.793 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:45.899 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:46.251 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:46.498 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:46.499 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:46.499 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:46.499 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:23:46.499 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:46.499 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:46.811 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-mirror 2024-04-26T00:23:47.326 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:08 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:47.448 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repo Size 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 3.1 M 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-04-26T00:23:47.449 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:47.450 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 3.1 M 2024-04-26T00:23:47.450 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 13 M 2024-04-26T00:23:47.450 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:48.587 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-mirror-19.0.0-2445.gd2c45d97.el9.x86_64.rpm 2.7 MB/s | 3.1 MB 00:01 2024-04-26T00:23:48.588 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:48.588 INFO:teuthology.orchestra.run.smithi072.stdout:Total 2.7 MB/s | 3.1 MB 00:01 2024-04-26T00:23:48.588 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:48.597 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:48.597 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:48.683 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:48.684 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:49.110 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:49.182 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:49.206 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:49.207 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T00:23:49.207 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T00:23:49.207 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T00:23:49.207 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T00:23:49.207 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:50.382 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:50.382 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:50.382 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:50.382 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:23:50.382 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:50.382 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:50.651 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install rbd-nbd 2024-04-26T00:23:51.173 INFO:teuthology.orchestra.run.smithi072.stdout:Last metadata expiration check: 0:02:12 ago on Fri 26 Apr 2024 12:21:39 AM UTC. 2024-04-26T00:23:51.296 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-04-26T00:23:51.297 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:51.297 INFO:teuthology.orchestra.run.smithi072.stdout: Package Architecture Version Repository Size 2024-04-26T00:23:51.297 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:51.297 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-04-26T00:23:51.297 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:19.0.0-2445.gd2c45d97.el9 ceph 171 k 2024-04-26T00:23:51.297 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:51.298 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-04-26T00:23:51.298 INFO:teuthology.orchestra.run.smithi072.stdout:================================================================================ 2024-04-26T00:23:51.298 INFO:teuthology.orchestra.run.smithi072.stdout:Install 1 Package 2024-04-26T00:23:51.298 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:51.298 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 171 k 2024-04-26T00:23:51.298 INFO:teuthology.orchestra.run.smithi072.stdout:Installed size: 494 k 2024-04-26T00:23:51.298 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-04-26T00:23:51.655 INFO:teuthology.orchestra.run.smithi072.stdout:rbd-nbd-19.0.0-2445.gd2c45d97.el9.x86_64.rpm 480 kB/s | 171 kB 00:00 2024-04-26T00:23:51.656 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-04-26T00:23:51.656 INFO:teuthology.orchestra.run.smithi072.stdout:Total 478 kB/s | 171 kB 00:00 2024-04-26T00:23:51.656 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-04-26T00:23:51.663 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-04-26T00:23:51.663 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-04-26T00:23:51.726 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-04-26T00:23:51.727 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-04-26T00:23:51.876 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-04-26T00:23:51.968 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:52.364 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:52.607 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 1/1 2024-04-26T00:23:52.607 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:52.607 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-04-26T00:23:52.607 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:19.0.0-2445.gd2c45d97.el9.x86_64 2024-04-26T00:23:52.607 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:52.607 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-04-26T00:23:52.879 DEBUG:teuthology.parallel:result is None 2024-04-26T00:23:52.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d2c45d977142e024a59b68a8f1dce8f0b9840551 2024-04-26T00:23:53.027 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T00:23:53.047 INFO:teuthology.orchestra.run.smithi072.stdout:19.0.0-2445.gd2c45d97.el9 2024-04-26T00:23:53.048 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2445.gd2c45d97.el9 2024-04-26T00:23:53.048 INFO:teuthology.task.install:The correct ceph version 19.0.0-2445.gd2c45d97 is installed. 2024-04-26T00:23:53.049 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T00:23:53.050 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:53.050 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T00:23:53.111 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T00:23:53.111 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:53.111 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T00:23:53.175 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T00:23:53.238 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T00:23:53.238 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:53.238 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T00:23:53.301 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T00:23:53.364 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T00:23:53.364 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:53.364 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T00:23:53.427 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T00:23:53.490 INFO:teuthology.run_tasks:Running task ceph... 2024-04-26T00:23:53.577 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-26T00:23:53.577 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /var/log/ceph 2024-04-26T00:23:53.600 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-26T00:23:53.600 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-26T00:23:53.663 INFO:tasks.ceph:Creating extra log directories... 2024-04-26T00:23:53.663 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-26T00:23:53.734 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-26T00:23:53.734 INFO:tasks.ceph:config {'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr2': False, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-04-26T00:23:53.734 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897', 'branch': 'squid', 'description': 'rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7673897', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr2': False, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'mon_bind_msgr2': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 9886, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'sleep_before_teardown': 0, 'subset': '30270/100000', 'suite': 'rados', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'targets': {'smithi072.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLeA4K+d8fVXbH4V0RktXOSYsn8DWHSfnGlr8UEnWBFfDYZ8KMV2Yn/l9COer6ugWFV7wCcZqQmruFMfRQ6FR0g='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', '\\(MON_DOWN\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr2': False, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug monc': 10, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'mon_bind_msgr2': False, 'sha1': 'd2c45d977142e024a59b68a8f1dce8f0b9840551', 'cluster': 'ceph'}}, {'rep_lost_unfound_delete': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_21:00:03', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-26T00:23:53.735 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-26T00:23:53.782 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m0777 -- /var/run/ceph 2024-04-26T00:23:53.845 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:53.846 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T00:23:53.901 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T00:23:53.902 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2024-04-26T00:23:53.955 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T00:23:53.956 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T00:23:53.956 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T00:23:53.956 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T00:23:53.956 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T00:23:53.956 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 00:23:49.470940927 +0000 2024-04-26T00:23:53.956 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 00:20:40.800893636 +0000 2024-04-26T00:23:53.956 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 00:20:40.800893636 +0000 2024-04-26T00:23:53.956 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-26 00:20:40.800893636 +0000 2024-04-26T00:23:53.956 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T00:23:54.017 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-26T00:23:54.017 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-26T00:23:54.017 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000204331 s, 2.5 MB/s 2024-04-26T00:23:54.019 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T00:23:54.074 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2024-04-26T00:23:54.128 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T00:23:54.129 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T00:23:54.129 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 681 Links: 1 2024-04-26T00:23:54.129 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T00:23:54.129 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T00:23:54.129 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 00:23:49.470940927 +0000 2024-04-26T00:23:54.129 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 00:20:41.137885787 +0000 2024-04-26T00:23:54.129 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 00:20:41.137885787 +0000 2024-04-26T00:23:54.129 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-26 00:20:41.137885787 +0000 2024-04-26T00:23:54.129 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T00:23:54.190 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-26T00:23:54.190 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-26T00:23:54.190 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000222881 s, 2.3 MB/s 2024-04-26T00:23:54.192 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T00:23:54.248 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2024-04-26T00:23:54.303 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T00:23:54.303 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T00:23:54.303 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-04-26T00:23:54.303 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T00:23:54.303 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T00:23:54.303 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 00:23:49.470940927 +0000 2024-04-26T00:23:54.303 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 00:20:41.494877472 +0000 2024-04-26T00:23:54.303 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 00:20:41.494877472 +0000 2024-04-26T00:23:54.304 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-26 00:20:41.494877472 +0000 2024-04-26T00:23:54.304 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T00:23:54.364 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-26T00:23:54.365 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-26T00:23:54.365 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000201007 s, 2.5 MB/s 2024-04-26T00:23:54.366 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T00:23:54.421 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-04-26 00:23:49.471940906 +0000 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-04-26 00:20:41.847869250 +0000 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-04-26 00:20:41.847869250 +0000 2024-04-26T00:23:54.476 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-04-26 00:20:41.847869250 +0000 2024-04-26T00:23:54.476 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T00:23:54.538 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-04-26T00:23:54.538 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-04-26T00:23:54.538 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000201321 s, 2.5 MB/s 2024-04-26T00:23:54.539 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T00:23:54.594 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-04-26T00:23:54.594 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-04-26T00:23:54.594 INFO:tasks.ceph:Generating config... 2024-04-26T00:23:54.595 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-26T00:23:54.596 INFO:tasks.ceph:[global] mon client hunt interval max multiple = 2 2024-04-26T00:23:54.596 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-04-26T00:23:54.596 INFO:tasks.ceph:[global] mon mgr beacon grace = 90 2024-04-26T00:23:54.596 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-04-26T00:23:54.596 INFO:tasks.ceph:[global] ms inject socket failures = 1000 2024-04-26T00:23:54.596 INFO:tasks.ceph:[global] ms type = async 2024-04-26T00:23:54.596 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-26T00:23:54.596 INFO:tasks.ceph:[mgr] debug monc = 10 2024-04-26T00:23:54.596 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-26T00:23:54.596 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-26T00:23:54.596 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-26T00:23:54.596 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-26T00:23:54.596 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-04-26T00:23:54.597 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-26T00:23:54.598 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-04-26T00:23:54.598 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-04-26T00:23:54.598 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-26T00:23:54.598 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-04-26T00:23:54.598 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-04-26T00:23:54.598 INFO:tasks.ceph:Setting up mon.a... 2024-04-26T00:23:54.598 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-26T00:23:54.675 INFO:teuthology.orchestra.run.smithi072.stdout:creating /etc/ceph/ceph.keyring 2024-04-26T00:23:54.678 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-26T00:23:54.758 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T00:23:54.821 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.72:6789]'), ('mon.b', '[v1:172.21.15.72:6790]'), ('mon.c', '[v1:172.21.15.72:6791]')] 2024-04-26T00:23:54.821 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.72:6789],[v1:172.21.15.72:6790],[v1:172.21.15.72:6791]', 'mon client directed command retry': 5, 'mon client hunt interval max multiple': 2, 'mon election default strategy': 3, 'mon mgr beacon grace': 90, 'ms bind msgr2': False, 'ms inject socket failures': 1000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug monc': 10}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-04-26T00:23:54.822 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:54.822 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T00:23:54.876 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.72:6789]' --addv b '[v1:172.21.15.72:6790]' --addv c '[v1:172.21.15.72:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: generated fsid fecf292c-395a-497e-b8b4-a892b12f3dd9 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:setting min_mon_release = quincy 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 0 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:fsid fecf292c-395a-497e-b8b4-a892b12f3dd9 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:last_changed 2024-04-26T00:23:54.984945+0000 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:created 2024-04-26T00:23:54.984945+0000 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:min_mon_release 17 (quincy) 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:election_strategy: 3 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:0: v1:172.21.15.72:6789/0 mon.a 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:1: v1:172.21.15.72:6790/0 mon.b 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:2: v1:172.21.15.72:6791/0 mon.c 2024-04-26T00:23:54.960 INFO:teuthology.orchestra.run.smithi072.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-26T00:23:54.962 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-26T00:23:54.976 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID fecf292c-395a-497e-b8b4-a892b12f3dd9... 2024-04-26T00:23:54.977 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout:[global] 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: chdir = "" 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: auth supported = cephx 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: filestore xattr use omap = true 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: mon clock drift allowed = 1.000 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: osd crush chooseleaf type = 0 2024-04-26T00:23:55.056 INFO:teuthology.orchestra.run.smithi072.stdout: auth debug = true 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on old message = true 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: ms die on bug = true 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: mon pg warn max object skew = 0 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: # disable pg_autoscaler by default for new pools 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default size = 2 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow primary affinity = true 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd allow pg remap = true 2024-04-26T00:23:55.057 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on legacy crush tunables = false 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on crush straw calc version zero = false 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on no sortbitwise = false 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on osd down out interval zero = false 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon warn on too few osds = false 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon_allow_pool_size_one = true 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: osd default data pool replay window = 5 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon allow pool delete = true 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.058 INFO:teuthology.orchestra.run.smithi072.stdout: mon cluster log file level = debug 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: debug asserts on shutdown = true 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: mon health detail to clog = false 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: mon host = "[v1:172.21.15.72:6789],[v1:172.21.15.72:6790],[v1:172.21.15.72:6791]" 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: mon client directed command retry = 5 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: mon client hunt interval max multiple = 2 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: mon election default strategy = 3 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: mon mgr beacon grace = 90 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: ms bind msgr2 = False 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: ms inject socket failures = 1000 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: ms type = async 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: fsid = fecf292c-395a-497e-b8b4-a892b12f3dd9 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout:[osd] 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: osd journal size = 100 2024-04-26T00:23:55.059 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub load threshold = 5.0 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd scrub max interval = 600 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd mclock profile = high_recovery_ops 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd recover clone overlap = true 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd recovery max chunk = 1048576 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug shutdown = true 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug op order = true 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify stray on activate = true 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug trim objects = true 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd open classes on start = true 2024-04-26T00:23:55.060 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug pg log writeout = true 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: osd deep scrub update digest min age = 30 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: osd map max advance = 10 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: journal zero on create = true 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: filestore ondisk finisher threads = 3 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: filestore apply finisher threads = 3 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: bdev debug aio = true 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug misdirected ops = true 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore block size = 96636764160 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore compression algorithm = lz4 2024-04-26T00:23:55.061 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore compression mode = aggressive 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore fsck on mount = True 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: bluestore zero block detection = True 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluefs = 20 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: debug bluestore = 20 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: debug osd = 20 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: debug rocksdb = 10 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd backfillfull_ratio = 0.85 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd full ratio = 0.9 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd nearfull ratio = 0.8 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify cached snaps = True 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: osd debug verify missing on start = True 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: osd failsafe full ratio = 0.95 2024-04-26T00:23:55.062 INFO:teuthology.orchestra.run.smithi072.stdout: osd mclock override recovery settings = True 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: osd objectstore = bluestore 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: osd op queue = debug_random 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: osd op queue cut off = debug_random 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr] 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: debug mgr = 20 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: mgr/telemetry/nag = false 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: debug monc = 10 2024-04-26T00:23:55.063 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout:[mon] 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout: debug ms = 1 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout: debug mon = 20 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout: debug paxos = 20 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout: debug auth = 20 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout: mon data avail warn = 5 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout: mon mgr mkfs grace = 240 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min pgs per osd = 4 2024-04-26T00:23:55.065 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd reporter subtree level = osd 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: mon osd prime pg temp = true 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: mon reweight min bytes per osd = 10 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: auth mon ticket ttl = 660 # 11m 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: auth service ticket ttl = 240 # 4m 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: # don't complain about insecure global_id in the test suite 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: # 1m isn't quite enough 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: mon_down_mkfs_grace = 2m 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.066 INFO:teuthology.orchestra.run.smithi072.stdout: mon_warn_on_filestore_osds = false 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout: mon scrub interval = 300 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout:[client] 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout: rgw cache enabled = true 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable ops log = true 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout: rgw enable usage log = true 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.a] 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.b] 2024-04-26T00:23:55.067 INFO:teuthology.orchestra.run.smithi072.stdout:[mon.c] 2024-04-26T00:23:55.068 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-26T00:23:55.068 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-26T00:23:55.160 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-26T00:23:55.160 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:55.160 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-26T00:23:55.175 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:55.175 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-26T00:23:55.229 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:23:55.229 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:55.229 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-26T00:23:55.229 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-26T00:23:55.301 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:23:55.301 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-26T00:23:55.356 INFO:tasks.ceph:Setting up mon nodes... 2024-04-26T00:23:55.356 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-26T00:23:55.356 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T00:23:55.444 INFO:teuthology.orchestra.run.smithi072.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-26T00:23:55.446 INFO:tasks.ceph:Setting up mds nodes... 2024-04-26T00:23:55.446 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-26T00:23:55.446 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-26T00:23:55.446 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi072.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-04-26T00:23:55.447 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-26T00:23:55.469 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-04-26T00:23:55.469 INFO:tasks.ceph:role: osd.0 2024-04-26T00:23:55.470 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:23:55.470 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-26T00:23:55.541 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T00:23:55.541 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T00:23:55.541 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T00:23:55.541 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T00:23:55.541 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T00:23:55.541 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2024-04-26T00:23:55.541 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T00:23:55.542 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T00:23:55.542 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T00:23:55.542 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T00:23:56.056 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2024-04-26T00:23:56.058 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2024-04-26T00:23:56.058 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-26T00:23:56.192 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-26T00:23:56.225 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-26T00:23:56.288 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-04-26T00:23:56.288 INFO:tasks.ceph:role: osd.1 2024-04-26T00:23:56.288 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:23:56.289 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-26T00:23:56.358 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T00:23:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T00:23:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T00:23:56.358 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T00:23:56.358 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T00:23:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2024-04-26T00:23:56.359 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T00:23:56.359 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T00:23:56.359 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T00:23:56.359 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T00:23:56.815 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2024-04-26T00:23:56.817 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2024-04-26T00:23:56.818 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-26T00:23:56.909 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-26T00:23:56.936 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-26T00:23:57.000 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-04-26T00:23:57.001 INFO:tasks.ceph:role: osd.2 2024-04-26T00:23:57.001 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:23:57.001 DEBUG:teuthology.orchestra.run.smithi072:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-26T00:23:57.072 INFO:teuthology.orchestra.run.smithi072.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-26T00:23:57.072 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-26T00:23:57.072 INFO:teuthology.orchestra.run.smithi072.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-26T00:23:57.072 INFO:teuthology.orchestra.run.smithi072.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-26T00:23:57.072 INFO:teuthology.orchestra.run.smithi072.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-26T00:23:57.072 INFO:teuthology.orchestra.run.smithi072.stdout: = sunit=0 swidth=0 blks 2024-04-26T00:23:57.072 INFO:teuthology.orchestra.run.smithi072.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-26T00:23:57.072 INFO:teuthology.orchestra.run.smithi072.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-26T00:23:57.073 INFO:teuthology.orchestra.run.smithi072.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-26T00:23:57.073 INFO:teuthology.orchestra.run.smithi072.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-26T00:23:57.559 INFO:teuthology.orchestra.run.smithi072.stdout:Discarding blocks...Done. 2024-04-26T00:23:57.561 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi072.front.sepia.ceph.com -o noatime 2024-04-26T00:23:57.561 DEBUG:teuthology.orchestra.run.smithi072:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-26T00:23:57.685 DEBUG:teuthology.orchestra.run.smithi072:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-26T00:23:57.715 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T00:23:57.819 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:23:57.843+0000 7fedcc4d7640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-26T00:23:57.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:23:57.844+0000 7fedcc4d7640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-26T00:23:57.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:23:57.845+0000 7fedcc4d7640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T00:23:57.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:23:57.845+0000 7fedcc4d7640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T00:23:57.821 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:23:57.845+0000 7fedcc4d7640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-26T00:23:57.829 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:23:57.853+0000 7fedcc4d7640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-26T00:24:00.673 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T00:24:00.701 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T00:24:00.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:00.812+0000 7f8ceb40f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-26T00:24:00.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:00.813+0000 7f8ceb40f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-26T00:24:00.789 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:00.813+0000 7f8ceb40f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T00:24:00.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:00.813+0000 7f8ceb40f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T00:24:00.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:00.813+0000 7f8ceb40f640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-26T00:24:00.790 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:00.813+0000 7f8ceb40f640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-26T00:24:03.632 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T00:24:03.658 DEBUG:teuthology.orchestra.run.smithi072:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-26T00:24:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:03.766+0000 7ffb2ba0e640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-26T00:24:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:03.766+0000 7ffb2ba0e640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-26T00:24:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:03.767+0000 7ffb2ba0e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T00:24:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:03.767+0000 7ffb2ba0e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T00:24:03.743 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:03.767+0000 7ffb2ba0e640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-26T00:24:03.744 INFO:teuthology.orchestra.run.smithi072.stderr:2024-04-26T00:24:03.767+0000 7ffb2ba0e640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-26T00:24:06.591 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T00:24:06.618 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-26T00:24:06.618 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:24:06.619 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-26T00:24:06.684 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:24:06.685 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-26T00:24:06.750 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:24:06.750 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-26T00:24:06.814 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:24:06.814 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-26T00:24:06.879 INFO:tasks.ceph:Adding keys to all mons... 2024-04-26T00:24:06.879 DEBUG:teuthology.orchestra.run.smithi072:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-26T00:24:06.942 INFO:teuthology.orchestra.run.smithi072.stdout:[mgr.x] 2024-04-26T00:24:06.942 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAb9CpmcXL0GxAAUZFrO32aUiTrLI/iojGyoQ== 2024-04-26T00:24:06.942 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.0] 2024-04-26T00:24:06.942 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAd9CpmRNJTMhAAVdTcDAfKZYMbD0Xt4GRX+A== 2024-04-26T00:24:06.942 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.1] 2024-04-26T00:24:06.942 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAg9Cpm5CCFMBAAA8JvDC55bwiAU9poVdk50Q== 2024-04-26T00:24:06.942 INFO:teuthology.orchestra.run.smithi072.stdout:[osd.2] 2024-04-26T00:24:06.942 INFO:teuthology.orchestra.run.smithi072.stdout: key = AQAj9Cpm72HFLRAAR2ynnCEFiI4lWOf6NcHrlw== 2024-04-26T00:24:06.943 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-26T00:24:07.033 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T00:24:07.080 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T00:24:07.128 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-26T00:24:07.175 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-26T00:24:07.175 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-26T00:24:07.198 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T00:24:07.291 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-26T00:24:07.313 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-26T00:24:07.377 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T00:24:07.470 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-26T00:24:07.493 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-26T00:24:07.557 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-26T00:24:07.650 DEBUG:teuthology.orchestra.run.smithi072:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-26T00:24:07.673 DEBUG:teuthology.orchestra.run.smithi072:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-26T00:24:07.727 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-26T00:24:07.727 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-26T00:24:07.727 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-26T00:24:07.771 INFO:tasks.ceph.mon.a:Started 2024-04-26T00:24:07.771 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-26T00:24:07.771 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-26T00:24:07.773 INFO:tasks.ceph.mon.b:Started 2024-04-26T00:24:07.774 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-26T00:24:07.774 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-26T00:24:07.776 INFO:tasks.ceph.mon.c:Started 2024-04-26T00:24:07.776 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph config set mgr mgr_pool false --force 2024-04-26T00:24:14.175 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-26T00:24:14.175 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-26T00:24:14.175 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-26T00:24:14.177 INFO:tasks.ceph.mgr.x:Started 2024-04-26T00:24:14.178 INFO:tasks.ceph:Setting crush tunables to default 2024-04-26T00:24:14.178 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd crush tunables default 2024-04-26T00:24:14.355 INFO:teuthology.orchestra.run.smithi072.stderr:adjusted tunables profile to default 2024-04-26T00:24:14.358 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:14.382+0000 7f64d7216040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T00:24:14.366 INFO:tasks.ceph:check_enable_crimson: False 2024-04-26T00:24:14.366 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-26T00:24:14.367 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:24:14.367 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-26T00:24:14.389 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:24:14.389 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-26T00:24:14.452 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-04-26T00:24:14.453 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-26T00:24:14.516 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 264b339b-fab7-4705-92f1-2b72519db1a7 0 2024-04-26T00:24:14.526 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:14.549+0000 7f64d7216040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T00:24:14.582 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:14.605+0000 7f64d7216040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T00:24:14.703 INFO:teuthology.orchestra.run.smithi072.stdout:0 2024-04-26T00:24:14.716 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new 200d0549-873b-4a17-b213-8c659dd090d5 1 2024-04-26T00:24:14.845 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:14.868+0000 7f64d7216040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T00:24:14.892 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:14.915+0000 7f64d7216040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T00:24:14.899 INFO:teuthology.orchestra.run.smithi072.stdout:1 2024-04-26T00:24:14.911 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd new f0d45389-e221-4b90-90d6-79aeafc0effd 2 2024-04-26T00:24:15.092 INFO:teuthology.orchestra.run.smithi072.stdout:2 2024-04-26T00:24:15.103 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T00:24:15.103 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-26T00:24:15.105 INFO:tasks.ceph.osd.0:Started 2024-04-26T00:24:15.105 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T00:24:15.106 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T00:24:15.107 INFO:tasks.ceph.osd.1:Started 2024-04-26T00:24:15.108 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T00:24:15.108 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T00:24:15.109 INFO:tasks.ceph.osd.2:Started 2024-04-26T00:24:15.110 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:24:15.310 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:15.310 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":5,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:24:15.112652+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T00:24:15.112652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:24:15.321 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-26T00:24:15.321 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-26T00:24:15.623 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T00:24:15.660 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:15.684+0000 7f64d7216040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T00:24:15.709 INFO:tasks.ceph.osd.2.smithi072.stderr:2024-04-26T00:24:15.732+0000 7f3b16c0e640 -1 Falling back to public interface 2024-04-26T00:24:15.709 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:24:15.733+0000 7fef16a21640 -1 Falling back to public interface 2024-04-26T00:24:15.724 INFO:tasks.ceph.osd.1.smithi072.stderr:2024-04-26T00:24:15.747+0000 7f686681a640 -1 Falling back to public interface 2024-04-26T00:24:15.754 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:15.777+0000 7f64d7216040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T00:24:15.781 INFO:teuthology.misc.health.smithi072.stdout: 2024-04-26T00:24:15.781 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":5,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:24:15.112652+0000","last_up_change":"0.000000","last_in_change":"2024-04-26T00:24:15.112652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:24:15.791 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-04-26T00:24:15.798 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:15.821+0000 7f64d7216040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T00:24:15.964 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:15.987+0000 7f64d7216040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T00:24:16.012 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:16.036+0000 7f64d7216040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T00:24:16.066 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:16.089+0000 7f64d7216040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T00:24:16.178 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:16.201+0000 7f64d7216040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T00:24:16.555 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:16.578+0000 7f64d7216040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T00:24:16.811 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:16.834+0000 7f64d7216040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T00:24:16.864 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:16.886+0000 7f64d7216040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T00:24:16.978 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:17.001+0000 7f64d7216040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T00:24:17.263 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:17.286+0000 7f64d7216040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T00:24:17.310 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:17.333+0000 7f64d7216040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T00:24:17.466 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:17.489+0000 7f64d7216040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T00:24:17.599 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:17.622+0000 7f64d7216040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T00:24:17.650 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:17.673+0000 7f64d7216040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T00:24:17.719 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:17.742+0000 7f64d7216040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T00:24:17.763 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:17.786+0000 7f64d7216040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T00:24:18.247 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:18.270+0000 7f64d7216040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T00:24:18.362 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:18.385+0000 7f64d7216040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T00:24:19.214 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:24:19.237+0000 7fef16a21640 -1 osd.0 0 log_to_monitors true 2024-04-26T00:24:19.230 INFO:tasks.ceph.osd.2.smithi072.stderr:2024-04-26T00:24:19.253+0000 7f3b16c0e640 -1 osd.2 0 log_to_monitors true 2024-04-26T00:24:19.245 INFO:tasks.ceph.osd.1.smithi072.stderr:2024-04-26T00:24:19.268+0000 7f686681a640 -1 osd.1 0 log_to_monitors true 2024-04-26T00:24:19.377 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:19.399+0000 7f6477b9b640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-26T00:24:19.377 INFO:tasks.ceph.mgr.x.smithi072.stderr:2024-04-26T00:24:19.399+0000 7f6477b9b640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-26T00:24:20.429 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:24:20.452+0000 7fef12488640 -1 osd.0 0 waiting for initial osdmap 2024-04-26T00:24:20.430 INFO:tasks.ceph.osd.2.smithi072.stderr:2024-04-26T00:24:20.453+0000 7f3b124cf640 -1 osd.2 0 waiting for initial osdmap 2024-04-26T00:24:20.437 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:24:20.460+0000 7fef0e2b0640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T00:24:20.437 INFO:tasks.ceph.osd.2.smithi072.stderr:2024-04-26T00:24:20.460+0000 7f3b0e2f7640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T00:24:20.970 INFO:tasks.ceph.osd.1.smithi072.stderr:2024-04-26T00:24:20.993+0000 7f6862180640 -1 osd.1 0 waiting for initial osdmap 2024-04-26T00:24:20.974 INFO:tasks.ceph.osd.1.smithi072.stderr:2024-04-26T00:24:20.996+0000 7f6859fa0640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T00:24:22.093 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-26T00:24:22.379 INFO:teuthology.misc.health.smithi072.stdout: 2024-04-26T00:24:22.379 INFO:teuthology.misc.health.smithi072.stdout:{"epoch":8,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:24:21.455748+0000","last_up_change":"2024-04-26T00:24:21.455748+0000","last_in_change":"2024-04-26T00:24:15.112652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":21990784}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":21990784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":21990784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":21990784}]},"public_addr":"172.21.15.72:6803/21990784","cluster_addr":"172.21.15.72:6805/21990784","heartbeat_back_addr":"172.21.15.72:6807/21990784","heartbeat_front_addr":"172.21.15.72:6806/21990784","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":1686523567}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":1686523567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":1686523567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":1686523567}]},"public_addr":"172.21.15.72:6808/1686523567","cluster_addr":"172.21.15.72:6809/1686523567","heartbeat_back_addr":"172.21.15.72:6811/1686523567","heartbeat_front_addr":"172.21.15.72:6810/1686523567","state":["exists","up"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":872899250}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":872899250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":872899250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":872899250}]},"public_addr":"172.21.15.72:6800/872899250","cluster_addr":"172.21.15.72:6801/872899250","heartbeat_back_addr":"172.21.15.72:6804/872899250","heartbeat_front_addr":"172.21.15.72:6802/872899250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:24:22.390 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-04-26T00:24:22.390 INFO:tasks.ceph:Creating RBD pool 2024-04-26T00:24:22.391 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-26T00:24:23.450 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'rbd' created 2024-04-26T00:24:23.470 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-26T00:24:24.459 INFO:teuthology.orchestra.run.smithi072.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-26T00:24:24.478 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-26T00:24:24.478 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T00:24:24.478 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-26T00:24:24.478 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-26T00:24:24.479 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-26T00:24:24.809 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:24.820 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":4,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6812","nonce":608264311}]},"active_addr":"172.21.15.72:6812/608264311","active_change":"2024-04-26T00:24:18.388104+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":1640703428}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":3233665076}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.72:0","nonce":2325167155}]}]} 2024-04-26T00:24:24.822 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-26T00:24:24.822 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-26T00:24:24.823 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:24:25.102 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:25.102 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":11,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:24:24.474970+0000","last_up_change":"2024-04-26T00:24:21.455748+0000","last_in_change":"2024-04-26T00:24:15.112652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":21990784}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":21990784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":21990784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":21990784}]},"public_addr":"172.21.15.72:6803/21990784","cluster_addr":"172.21.15.72:6805/21990784","heartbeat_back_addr":"172.21.15.72:6807/21990784","heartbeat_front_addr":"172.21.15.72:6806/21990784","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":1686523567}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":1686523567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":1686523567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":1686523567}]},"public_addr":"172.21.15.72:6808/1686523567","cluster_addr":"172.21.15.72:6809/1686523567","heartbeat_back_addr":"172.21.15.72:6811/1686523567","heartbeat_front_addr":"172.21.15.72:6810/1686523567","state":["exists","up"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":872899250}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":872899250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":872899250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":872899250}]},"public_addr":"172.21.15.72:6800/872899250","cluster_addr":"172.21.15.72:6801/872899250","heartbeat_back_addr":"172.21.15.72:6804/872899250","heartbeat_front_addr":"172.21.15.72:6802/872899250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:24:25.114 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-26T00:24:25.114 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:24:25.392 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:25.393 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":11,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:24:24.474970+0000","last_up_change":"2024-04-26T00:24:21.455748+0000","last_in_change":"2024-04-26T00:24:15.112652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":21990784}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":21990784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":21990784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":21990784}]},"public_addr":"172.21.15.72:6803/21990784","cluster_addr":"172.21.15.72:6805/21990784","heartbeat_back_addr":"172.21.15.72:6807/21990784","heartbeat_front_addr":"172.21.15.72:6806/21990784","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":1686523567}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":1686523567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":1686523567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":1686523567}]},"public_addr":"172.21.15.72:6808/1686523567","cluster_addr":"172.21.15.72:6809/1686523567","heartbeat_back_addr":"172.21.15.72:6811/1686523567","heartbeat_front_addr":"172.21.15.72:6810/1686523567","state":["exists","up"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":872899250}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":872899250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":872899250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":872899250}]},"public_addr":"172.21.15.72:6800/872899250","cluster_addr":"172.21.15.72:6801/872899250","heartbeat_back_addr":"172.21.15.72:6804/872899250","heartbeat_front_addr":"172.21.15.72:6802/872899250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:24:25.405 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T00:24:25.405 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T00:24:25.406 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T00:24:25.541 INFO:teuthology.orchestra.run.smithi072.stdout:34359738370 2024-04-26T00:24:25.542 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:24:25.563 INFO:teuthology.orchestra.run.smithi072.stdout:34359738371 2024-04-26T00:24:25.563 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:25.564 INFO:teuthology.orchestra.run.smithi072.stdout:34359738370 2024-04-26T00:24:25.564 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:25.853 INFO:teuthology.orchestra.run.smithi072.stdout:34359738369 2024-04-26T00:24:25.867 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738369 for osd.2 2024-04-26T00:24:25.900 INFO:teuthology.orchestra.run.smithi072.stdout:34359738370 2024-04-26T00:24:25.913 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738370 for osd.1 2024-04-26T00:24:25.918 INFO:teuthology.orchestra.run.smithi072.stdout:34359738369 2024-04-26T00:24:25.929 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738369 for osd.0 2024-04-26T00:24:26.867 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:24:26.913 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:26.930 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:27.194 INFO:teuthology.orchestra.run.smithi072.stdout:34359738370 2024-04-26T00:24:27.207 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738370 for osd.2 2024-04-26T00:24:27.207 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:27.241 INFO:teuthology.orchestra.run.smithi072.stdout:34359738371 2024-04-26T00:24:27.253 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738371 got 34359738371 for osd.1 2024-04-26T00:24:27.254 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:27.270 INFO:teuthology.orchestra.run.smithi072.stdout:34359738370 2024-04-26T00:24:27.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738370 got 34359738370 for osd.0 2024-04-26T00:24:27.283 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:27.283 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T00:24:27.283 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:27.602 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:27.602 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:27.613 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-04-26T00:24:26.396650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81364,"kb_used_data":288,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034156,"statfs":{"total":289910292480,"available":289826975744,"internally_reserved":0,"allocated":294912,"data_stored":64173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492047+0000","last_change":"2024-04-26T00:24:24.488255+0000","last_active":"2024-04-26T00:24:25.492047+0000","last_peered":"2024-04-26T00:24:25.492047+0000","last_clean":"2024-04-26T00:24:25.492047+0000","last_became_active":"2024-04-26T00:24:24.487963+0000","last_became_peered":"2024-04-26T00:24:24.487963+0000","last_unstale":"2024-04-26T00:24:25.492047+0000","last_undegraded":"2024-04-26T00:24:25.492047+0000","last_fullsized":"2024-04-26T00:24:25.492047+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:24:52.082080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492073+0000","last_change":"2024-04-26T00:24:24.485307+0000","last_active":"2024-04-26T00:24:25.492073+0000","last_peered":"2024-04-26T00:24:25.492073+0000","last_clean":"2024-04-26T00:24:25.492073+0000","last_became_active":"2024-04-26T00:24:24.484577+0000","last_became_peered":"2024-04-26T00:24:24.484577+0000","last_unstale":"2024-04-26T00:24:25.492073+0000","last_undegraded":"2024-04-26T00:24:25.492073+0000","last_fullsized":"2024-04-26T00:24:25.492073+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:22:47.688275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-26T00:24:24.491124+0000","last_change":"2024-04-26T00:24:24.489032+0000","last_active":"2024-04-26T00:24:24.491124+0000","last_peered":"2024-04-26T00:24:24.491124+0000","last_clean":"2024-04-26T00:24:24.491124+0000","last_became_active":"2024-04-26T00:24:24.488722+0000","last_became_peered":"2024-04-26T00:24:24.488722+0000","last_unstale":"2024-04-26T00:24:24.491124+0000","last_undegraded":"2024-04-26T00:24:24.491124+0000","last_fullsized":"2024-04-26T00:24:24.491124+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:08.054687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492153+0000","last_change":"2024-04-26T00:24:24.484819+0000","last_active":"2024-04-26T00:24:25.492153+0000","last_peered":"2024-04-26T00:24:25.492153+0000","last_clean":"2024-04-26T00:24:25.492153+0000","last_became_active":"2024-04-26T00:24:24.484550+0000","last_became_peered":"2024-04-26T00:24:24.484550+0000","last_unstale":"2024-04-26T00:24:25.492153+0000","last_undegraded":"2024-04-26T00:24:25.492153+0000","last_fullsized":"2024-04-26T00:24:25.492153+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:24.731073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.491828+0000","last_change":"2024-04-26T00:24:24.487359+0000","last_active":"2024-04-26T00:24:25.491828+0000","last_peered":"2024-04-26T00:24:25.491828+0000","last_clean":"2024-04-26T00:24:25.491828+0000","last_became_active":"2024-04-26T00:24:24.486618+0000","last_became_peered":"2024-04-26T00:24:24.486618+0000","last_unstale":"2024-04-26T00:24:25.491828+0000","last_undegraded":"2024-04-26T00:24:25.491828+0000","last_fullsized":"2024-04-26T00:24:25.491828+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:27:23.779539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.490710+0000","last_change":"2024-04-26T00:24:24.487504+0000","last_active":"2024-04-26T00:24:25.490710+0000","last_peered":"2024-04-26T00:24:25.490710+0000","last_clean":"2024-04-26T00:24:25.490710+0000","last_became_active":"2024-04-26T00:24:24.486992+0000","last_became_peered":"2024-04-26T00:24:24.486992+0000","last_unstale":"2024-04-26T00:24:25.490710+0000","last_undegraded":"2024-04-26T00:24:25.490710+0000","last_fullsized":"2024-04-26T00:24:25.490710+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-26T00:24:24.491154+0000","last_change":"2024-04-26T00:24:24.489079+0000","last_active":"2024-04-26T00:24:24.491154+0000","last_peered":"2024-04-26T00:24:24.491154+0000","last_clean":"2024-04-26T00:24:24.491154+0000","last_became_active":"2024-04-26T00:24:24.488765+0000","last_became_peered":"2024-04-26T00:24:24.488765+0000","last_unstale":"2024-04-26T00:24:24.491154+0000","last_undegraded":"2024-04-26T00:24:24.491154+0000","last_fullsized":"2024-04-26T00:24:24.491154+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:41.844001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.491849+0000","last_change":"2024-04-26T00:24:24.484927+0000","last_active":"2024-04-26T00:24:25.491849+0000","last_peered":"2024-04-26T00:24:25.491849+0000","last_clean":"2024-04-26T00:24:25.491849+0000","last_became_active":"2024-04-26T00:24:24.484616+0000","last_became_peered":"2024-04-26T00:24:24.484616+0000","last_unstale":"2024-04-26T00:24:25.491849+0000","last_undegraded":"2024-04-26T00:24:25.491849+0000","last_fullsized":"2024-04-26T00:24:25.491849+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:37.127557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":98304,"data_stored":21391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738370,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":98304,"data_stored":21391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738370,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":98304,"data_stored":21391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:27.614 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:27.889 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:27.890 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:27.901 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":12,"stamp":"2024-04-26T00:24:26.396650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81364,"kb_used_data":288,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034156,"statfs":{"total":289910292480,"available":289826975744,"internally_reserved":0,"allocated":294912,"data_stored":64173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492047+0000","last_change":"2024-04-26T00:24:24.488255+0000","last_active":"2024-04-26T00:24:25.492047+0000","last_peered":"2024-04-26T00:24:25.492047+0000","last_clean":"2024-04-26T00:24:25.492047+0000","last_became_active":"2024-04-26T00:24:24.487963+0000","last_became_peered":"2024-04-26T00:24:24.487963+0000","last_unstale":"2024-04-26T00:24:25.492047+0000","last_undegraded":"2024-04-26T00:24:25.492047+0000","last_fullsized":"2024-04-26T00:24:25.492047+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:24:52.082080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492073+0000","last_change":"2024-04-26T00:24:24.485307+0000","last_active":"2024-04-26T00:24:25.492073+0000","last_peered":"2024-04-26T00:24:25.492073+0000","last_clean":"2024-04-26T00:24:25.492073+0000","last_became_active":"2024-04-26T00:24:24.484577+0000","last_became_peered":"2024-04-26T00:24:24.484577+0000","last_unstale":"2024-04-26T00:24:25.492073+0000","last_undegraded":"2024-04-26T00:24:25.492073+0000","last_fullsized":"2024-04-26T00:24:25.492073+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:22:47.688275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-26T00:24:24.491124+0000","last_change":"2024-04-26T00:24:24.489032+0000","last_active":"2024-04-26T00:24:24.491124+0000","last_peered":"2024-04-26T00:24:24.491124+0000","last_clean":"2024-04-26T00:24:24.491124+0000","last_became_active":"2024-04-26T00:24:24.488722+0000","last_became_peered":"2024-04-26T00:24:24.488722+0000","last_unstale":"2024-04-26T00:24:24.491124+0000","last_undegraded":"2024-04-26T00:24:24.491124+0000","last_fullsized":"2024-04-26T00:24:24.491124+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:08.054687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492153+0000","last_change":"2024-04-26T00:24:24.484819+0000","last_active":"2024-04-26T00:24:25.492153+0000","last_peered":"2024-04-26T00:24:25.492153+0000","last_clean":"2024-04-26T00:24:25.492153+0000","last_became_active":"2024-04-26T00:24:24.484550+0000","last_became_peered":"2024-04-26T00:24:24.484550+0000","last_unstale":"2024-04-26T00:24:25.492153+0000","last_undegraded":"2024-04-26T00:24:25.492153+0000","last_fullsized":"2024-04-26T00:24:25.492153+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:24.731073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.491828+0000","last_change":"2024-04-26T00:24:24.487359+0000","last_active":"2024-04-26T00:24:25.491828+0000","last_peered":"2024-04-26T00:24:25.491828+0000","last_clean":"2024-04-26T00:24:25.491828+0000","last_became_active":"2024-04-26T00:24:24.486618+0000","last_became_peered":"2024-04-26T00:24:24.486618+0000","last_unstale":"2024-04-26T00:24:25.491828+0000","last_undegraded":"2024-04-26T00:24:25.491828+0000","last_fullsized":"2024-04-26T00:24:25.491828+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:27:23.779539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.490710+0000","last_change":"2024-04-26T00:24:24.487504+0000","last_active":"2024-04-26T00:24:25.490710+0000","last_peered":"2024-04-26T00:24:25.490710+0000","last_clean":"2024-04-26T00:24:25.490710+0000","last_became_active":"2024-04-26T00:24:24.486992+0000","last_became_peered":"2024-04-26T00:24:24.486992+0000","last_unstale":"2024-04-26T00:24:25.490710+0000","last_undegraded":"2024-04-26T00:24:25.490710+0000","last_fullsized":"2024-04-26T00:24:25.490710+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-26T00:24:24.491154+0000","last_change":"2024-04-26T00:24:24.489079+0000","last_active":"2024-04-26T00:24:24.491154+0000","last_peered":"2024-04-26T00:24:24.491154+0000","last_clean":"2024-04-26T00:24:24.491154+0000","last_became_active":"2024-04-26T00:24:24.488765+0000","last_became_peered":"2024-04-26T00:24:24.488765+0000","last_unstale":"2024-04-26T00:24:24.491154+0000","last_undegraded":"2024-04-26T00:24:24.491154+0000","last_fullsized":"2024-04-26T00:24:24.491154+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:09:41.844001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.491849+0000","last_change":"2024-04-26T00:24:24.484927+0000","last_active":"2024-04-26T00:24:25.491849+0000","last_peered":"2024-04-26T00:24:25.491849+0000","last_clean":"2024-04-26T00:24:25.491849+0000","last_became_active":"2024-04-26T00:24:24.484616+0000","last_became_peered":"2024-04-26T00:24:24.484616+0000","last_unstale":"2024-04-26T00:24:25.491849+0000","last_undegraded":"2024-04-26T00:24:25.491849+0000","last_fullsized":"2024-04-26T00:24:25.491849+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:37.127557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738371,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":98304,"data_stored":21391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738370,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":98304,"data_stored":21391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738370,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":98304,"data_stored":21391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:27.902 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T00:24:27.902 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T00:24:27.902 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-26T00:24:27.902 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-26T00:24:28.216 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:28.216 INFO:teuthology.orchestra.run.smithi072.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T00:24:28.228 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-26T00:24:28.229 INFO:teuthology.run_tasks:Running task rep_lost_unfound_delete... 2024-04-26T00:24:28.254 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:24:28.539 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:28.539 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":12,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:24:25.479476+0000","last_up_change":"2024-04-26T00:24:21.455748+0000","last_in_change":"2024-04-26T00:24:15.112652+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":21990784}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":21990784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":21990784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":21990784}]},"public_addr":"172.21.15.72:6803/21990784","cluster_addr":"172.21.15.72:6805/21990784","heartbeat_back_addr":"172.21.15.72:6807/21990784","heartbeat_front_addr":"172.21.15.72:6806/21990784","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":1686523567}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":1686523567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":1686523567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":1686523567}]},"public_addr":"172.21.15.72:6808/1686523567","cluster_addr":"172.21.15.72:6809/1686523567","heartbeat_back_addr":"172.21.15.72:6811/1686523567","heartbeat_front_addr":"172.21.15.72:6810/1686523567","state":["exists","up"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":872899250}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":872899250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":872899250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":872899250}]},"public_addr":"172.21.15.72:6800/872899250","cluster_addr":"172.21.15.72:6801/872899250","heartbeat_back_addr":"172.21.15.72:6804/872899250","heartbeat_front_addr":"172.21.15.72:6802/872899250","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:24:28.551 INFO:tasks.rep_lost_unfound_delete.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-26T00:24:22.712820+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '12', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T00:24:28.552 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-04-26T00:24:28.830 INFO:teuthology.orchestra.run.smithi072.stdout:pg_num: 8 2024-04-26T00:24:28.843 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:epoch 12 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:fsid fecf292c-395a-497e-b8b4-a892b12f3dd9 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:created 2024-04-26T00:24:14.092045+0000 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:modified 2024-04-26T00:24:25.479476+0000 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:crush_version 4 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:full_ratio 0.95 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:backfillfull_ratio 0.9 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:nearfull_ratio 0.85 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:require_min_compat_client luminous 2024-04-26T00:24:29.123 INFO:teuthology.orchestra.run.smithi072.stdout:min_compat_client jewel 2024-04-26T00:24:29.124 INFO:teuthology.orchestra.run.smithi072.stdout:require_osd_release squid 2024-04-26T00:24:29.124 INFO:teuthology.orchestra.run.smithi072.stdout:stretch_mode_enabled false 2024-04-26T00:24:29.124 INFO:teuthology.orchestra.run.smithi072.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 12 flags hashpspool stripe_width 0 application rbd read_balance_score 1.88 2024-04-26T00:24:29.124 INFO:teuthology.orchestra.run.smithi072.stdout:max_osd 3 2024-04-26T00:24:29.124 INFO:teuthology.orchestra.run.smithi072.stdout:osd.0 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6803/21990784 v1:172.21.15.72:6805/21990784 exists,up 264b339b-fab7-4705-92f1-2b72519db1a7 2024-04-26T00:24:29.124 INFO:teuthology.orchestra.run.smithi072.stdout:osd.1 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6808/1686523567 v1:172.21.15.72:6809/1686523567 exists,up 200d0549-873b-4a17-b213-8c659dd090d5 2024-04-26T00:24:29.124 INFO:teuthology.orchestra.run.smithi072.stdout:osd.2 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6800/872899250 v1:172.21.15.72:6801/872899250 exists,up f0d45389-e221-4b90-90d6-79aeafc0effd 2024-04-26T00:24:29.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:['osd.0 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6803/21990784 v1:172.21.15.72:6805/21990784 exists,up 264b339b-fab7-4705-92f1-2b72519db1a7', 'osd.1 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6808/1686523567 v1:172.21.15.72:6809/1686523567 exists,up 200d0549-873b-4a17-b213-8c659dd090d5', 'osd.2 up in weight 1 up_from 8 up_thru 10 down_at 0 last_clean_interval [0,0) v1:172.21.15.72:6800/872899250 v1:172.21.15.72:6801/872899250 exists,up f0d45389-e221-4b90-90d6-79aeafc0effd'] 2024-04-26T00:24:29.136 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T00:24:29.136 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T00:24:29.137 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T00:24:29.271 INFO:teuthology.orchestra.run.smithi072.stdout:34359738372 2024-04-26T00:24:29.271 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:24:29.294 INFO:teuthology.orchestra.run.smithi072.stdout:34359738373 2024-04-26T00:24:29.295 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:29.295 INFO:teuthology.orchestra.run.smithi072.stdout:34359738373 2024-04-26T00:24:29.295 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:29.576 INFO:teuthology.orchestra.run.smithi072.stdout:34359738371 2024-04-26T00:24:29.588 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738372 got 34359738371 for osd.2 2024-04-26T00:24:29.632 INFO:teuthology.orchestra.run.smithi072.stdout:34359738371 2024-04-26T00:24:29.634 INFO:teuthology.orchestra.run.smithi072.stdout:34359738370 2024-04-26T00:24:29.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738371 for osd.1 2024-04-26T00:24:29.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738370 for osd.0 2024-04-26T00:24:30.589 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:24:30.646 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:30.647 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:30.940 INFO:teuthology.orchestra.run.smithi072.stdout:34359738372 2024-04-26T00:24:30.947 INFO:teuthology.orchestra.run.smithi072.stdout:34359738373 2024-04-26T00:24:30.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738372 got 34359738372 for osd.2 2024-04-26T00:24:30.956 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:30.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738373 for osd.0 2024-04-26T00:24:30.961 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:30.982 INFO:teuthology.orchestra.run.smithi072.stdout:34359738373 2024-04-26T00:24:30.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738373 for osd.1 2024-04-26T00:24:30.995 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:30.996 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-04-26T00:24:30.996 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:31.311 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:31.311 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:31.323 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-04-26T00:24:30.397224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81376,"kb_used_data":312,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034144,"statfs":{"total":289910292480,"available":289826963456,"internally_reserved":0,"allocated":319488,"data_stored":72189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492047+0000","last_change":"2024-04-26T00:24:24.488255+0000","last_active":"2024-04-26T00:24:25.492047+0000","last_peered":"2024-04-26T00:24:25.492047+0000","last_clean":"2024-04-26T00:24:25.492047+0000","last_became_active":"2024-04-26T00:24:24.487963+0000","last_became_peered":"2024-04-26T00:24:24.487963+0000","last_unstale":"2024-04-26T00:24:25.492047+0000","last_undegraded":"2024-04-26T00:24:25.492047+0000","last_fullsized":"2024-04-26T00:24:25.492047+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:24:52.082080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492073+0000","last_change":"2024-04-26T00:24:24.485307+0000","last_active":"2024-04-26T00:24:25.492073+0000","last_peered":"2024-04-26T00:24:25.492073+0000","last_clean":"2024-04-26T00:24:25.492073+0000","last_became_active":"2024-04-26T00:24:24.484577+0000","last_became_peered":"2024-04-26T00:24:24.484577+0000","last_unstale":"2024-04-26T00:24:25.492073+0000","last_undegraded":"2024-04-26T00:24:25.492073+0000","last_fullsized":"2024-04-26T00:24:25.492073+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:22:47.688275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:26.257554+0000","last_change":"2024-04-26T00:24:24.489032+0000","last_active":"2024-04-26T00:24:26.257554+0000","last_peered":"2024-04-26T00:24:26.257554+0000","last_clean":"2024-04-26T00:24:26.257554+0000","last_became_active":"2024-04-26T00:24:24.488722+0000","last_became_peered":"2024-04-26T00:24:24.488722+0000","last_unstale":"2024-04-26T00:24:26.257554+0000","last_undegraded":"2024-04-26T00:24:26.257554+0000","last_fullsized":"2024-04-26T00:24:26.257554+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:32.040189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492153+0000","last_change":"2024-04-26T00:24:24.484819+0000","last_active":"2024-04-26T00:24:25.492153+0000","last_peered":"2024-04-26T00:24:25.492153+0000","last_clean":"2024-04-26T00:24:25.492153+0000","last_became_active":"2024-04-26T00:24:24.484550+0000","last_became_peered":"2024-04-26T00:24:24.484550+0000","last_unstale":"2024-04-26T00:24:25.492153+0000","last_undegraded":"2024-04-26T00:24:25.492153+0000","last_fullsized":"2024-04-26T00:24:25.492153+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:24.731073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.491828+0000","last_change":"2024-04-26T00:24:24.487359+0000","last_active":"2024-04-26T00:24:25.491828+0000","last_peered":"2024-04-26T00:24:25.491828+0000","last_clean":"2024-04-26T00:24:25.491828+0000","last_became_active":"2024-04-26T00:24:24.486618+0000","last_became_peered":"2024-04-26T00:24:24.486618+0000","last_unstale":"2024-04-26T00:24:25.491828+0000","last_undegraded":"2024-04-26T00:24:25.491828+0000","last_fullsized":"2024-04-26T00:24:25.491828+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:27:23.779539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.490710+0000","last_change":"2024-04-26T00:24:24.487504+0000","last_active":"2024-04-26T00:24:25.490710+0000","last_peered":"2024-04-26T00:24:25.490710+0000","last_clean":"2024-04-26T00:24:25.490710+0000","last_became_active":"2024-04-26T00:24:24.486992+0000","last_became_peered":"2024-04-26T00:24:24.486992+0000","last_unstale":"2024-04-26T00:24:25.490710+0000","last_undegraded":"2024-04-26T00:24:25.490710+0000","last_fullsized":"2024-04-26T00:24:25.490710+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:26.257545+0000","last_change":"2024-04-26T00:24:24.489079+0000","last_active":"2024-04-26T00:24:26.257545+0000","last_peered":"2024-04-26T00:24:26.257545+0000","last_clean":"2024-04-26T00:24:26.257545+0000","last_became_active":"2024-04-26T00:24:24.488765+0000","last_became_peered":"2024-04-26T00:24:24.488765+0000","last_unstale":"2024-04-26T00:24:26.257545+0000","last_undegraded":"2024-04-26T00:24:26.257545+0000","last_fullsized":"2024-04-26T00:24:26.257545+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:50:27.973073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.491849+0000","last_change":"2024-04-26T00:24:24.484927+0000","last_active":"2024-04-26T00:24:25.491849+0000","last_peered":"2024-04-26T00:24:25.491849+0000","last_clean":"2024-04-26T00:24:25.491849+0000","last_became_active":"2024-04-26T00:24:24.484616+0000","last_became_peered":"2024-04-26T00:24:24.484616+0000","last_unstale":"2024-04-26T00:24:25.491849+0000","last_undegraded":"2024-04-26T00:24:25.491849+0000","last_fullsized":"2024-04-26T00:24:25.491849+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:37.127557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":106496,"data_stored":24063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":106496,"data_stored":24063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":106496,"data_stored":24063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:31.324 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:31.600 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:31.601 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:31.612 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2024-04-26T00:24:30.397224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81376,"kb_used_data":312,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034144,"statfs":{"total":289910292480,"available":289826963456,"internally_reserved":0,"allocated":319488,"data_stored":72189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492047+0000","last_change":"2024-04-26T00:24:24.488255+0000","last_active":"2024-04-26T00:24:25.492047+0000","last_peered":"2024-04-26T00:24:25.492047+0000","last_clean":"2024-04-26T00:24:25.492047+0000","last_became_active":"2024-04-26T00:24:24.487963+0000","last_became_peered":"2024-04-26T00:24:24.487963+0000","last_unstale":"2024-04-26T00:24:25.492047+0000","last_undegraded":"2024-04-26T00:24:25.492047+0000","last_fullsized":"2024-04-26T00:24:25.492047+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:24:52.082080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492073+0000","last_change":"2024-04-26T00:24:24.485307+0000","last_active":"2024-04-26T00:24:25.492073+0000","last_peered":"2024-04-26T00:24:25.492073+0000","last_clean":"2024-04-26T00:24:25.492073+0000","last_became_active":"2024-04-26T00:24:24.484577+0000","last_became_peered":"2024-04-26T00:24:24.484577+0000","last_unstale":"2024-04-26T00:24:25.492073+0000","last_undegraded":"2024-04-26T00:24:25.492073+0000","last_fullsized":"2024-04-26T00:24:25.492073+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:22:47.688275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:26.257554+0000","last_change":"2024-04-26T00:24:24.489032+0000","last_active":"2024-04-26T00:24:26.257554+0000","last_peered":"2024-04-26T00:24:26.257554+0000","last_clean":"2024-04-26T00:24:26.257554+0000","last_became_active":"2024-04-26T00:24:24.488722+0000","last_became_peered":"2024-04-26T00:24:24.488722+0000","last_unstale":"2024-04-26T00:24:26.257554+0000","last_undegraded":"2024-04-26T00:24:26.257554+0000","last_fullsized":"2024-04-26T00:24:26.257554+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:48:32.040189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.492153+0000","last_change":"2024-04-26T00:24:24.484819+0000","last_active":"2024-04-26T00:24:25.492153+0000","last_peered":"2024-04-26T00:24:25.492153+0000","last_clean":"2024-04-26T00:24:25.492153+0000","last_became_active":"2024-04-26T00:24:24.484550+0000","last_became_peered":"2024-04-26T00:24:24.484550+0000","last_unstale":"2024-04-26T00:24:25.492153+0000","last_undegraded":"2024-04-26T00:24:25.492153+0000","last_fullsized":"2024-04-26T00:24:25.492153+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:24.731073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.491828+0000","last_change":"2024-04-26T00:24:24.487359+0000","last_active":"2024-04-26T00:24:25.491828+0000","last_peered":"2024-04-26T00:24:25.491828+0000","last_clean":"2024-04-26T00:24:25.491828+0000","last_became_active":"2024-04-26T00:24:24.486618+0000","last_became_peered":"2024-04-26T00:24:24.486618+0000","last_unstale":"2024-04-26T00:24:25.491828+0000","last_undegraded":"2024-04-26T00:24:25.491828+0000","last_fullsized":"2024-04-26T00:24:25.491828+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:27:23.779539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.490710+0000","last_change":"2024-04-26T00:24:24.487504+0000","last_active":"2024-04-26T00:24:25.490710+0000","last_peered":"2024-04-26T00:24:25.490710+0000","last_clean":"2024-04-26T00:24:25.490710+0000","last_became_active":"2024-04-26T00:24:24.486992+0000","last_became_peered":"2024-04-26T00:24:24.486992+0000","last_unstale":"2024-04-26T00:24:25.490710+0000","last_undegraded":"2024-04-26T00:24:25.490710+0000","last_fullsized":"2024-04-26T00:24:25.490710+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:26.257545+0000","last_change":"2024-04-26T00:24:24.489079+0000","last_active":"2024-04-26T00:24:26.257545+0000","last_peered":"2024-04-26T00:24:26.257545+0000","last_clean":"2024-04-26T00:24:26.257545+0000","last_became_active":"2024-04-26T00:24:24.488765+0000","last_became_peered":"2024-04-26T00:24:24.488765+0000","last_unstale":"2024-04-26T00:24:26.257545+0000","last_undegraded":"2024-04-26T00:24:26.257545+0000","last_fullsized":"2024-04-26T00:24:26.257545+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:50:27.973073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-26T00:24:25.491849+0000","last_change":"2024-04-26T00:24:24.484927+0000","last_active":"2024-04-26T00:24:25.491849+0000","last_peered":"2024-04-26T00:24:25.491849+0000","last_clean":"2024-04-26T00:24:25.491849+0000","last_became_active":"2024-04-26T00:24:24.484616+0000","last_became_peered":"2024-04-26T00:24:24.484616+0000","last_unstale":"2024-04-26T00:24:25.491849+0000","last_undegraded":"2024-04-26T00:24:25.491849+0000","last_fullsized":"2024-04-26T00:24:25.491849+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:37.127557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27128,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344712,"statfs":{"total":96636764160,"available":96608985088,"internally_reserved":0,"allocated":106496,"data_stored":24063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":106496,"data_stored":24063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":104,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":106496,"data_stored":24063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:31.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-04-26T00:24:31.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:creating pool_name unfounddel_pool 2024-04-26T00:24:31.614 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfounddel_pool 16 2024-04-26T00:24:32.487 INFO:teuthology.orchestra.run.smithi072.stderr:pool 'unfounddel_pool' created 2024-04-26T00:24:32.506 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfounddel_pool rados --yes-i-really-mean-it || true 2024-04-26T00:24:33.491 INFO:teuthology.orchestra.run.smithi072.stderr:enabled application 'rados' on pool 'unfounddel_pool' 2024-04-26T00:24:34.517 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T00:24:34.517 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:24:34.550 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T00:24:34.550 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-04-26T00:24:34.890 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:24:34.913+0000 7ffb0b211640 -1 mon.a@0(leader).osd e15 definitely_dead 0 2024-04-26T00:24:35.510 INFO:teuthology.orchestra.run.smithi072.stderr:marked down osd.2. 2024-04-26T00:24:35.527 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-04-26T00:24:35.817 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:24:35.839+0000 7ffb0b211640 -1 mon.a@0(leader).osd e16 definitely_dead 0 2024-04-26T00:24:36.518 INFO:teuthology.orchestra.run.smithi072.stderr:marked out osd.2. 2024-04-26T00:24:36.534 INFO:tasks.util.rados:rados -p unfounddel_pool put dummy /etc/fstab 2024-04-26T00:24:36.534 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put dummy /etc/fstab 2024-04-26T00:24:36.584 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T00:24:36.585 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T00:24:36.718 INFO:teuthology.orchestra.run.smithi072.stdout:34359738375 2024-04-26T00:24:36.719 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:36.720 INFO:teuthology.orchestra.run.smithi072.stdout:34359738375 2024-04-26T00:24:36.720 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:37.054 INFO:teuthology.orchestra.run.smithi072.stdout:34359738374 2024-04-26T00:24:37.054 INFO:teuthology.orchestra.run.smithi072.stdout:34359738374 2024-04-26T00:24:37.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738374 for osd.1 2024-04-26T00:24:37.068 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738374 for osd.0 2024-04-26T00:24:38.068 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:38.069 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:38.377 INFO:teuthology.orchestra.run.smithi072.stdout:34359738374 2024-04-26T00:24:38.378 INFO:teuthology.orchestra.run.smithi072.stdout:34359738374 2024-04-26T00:24:38.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738374 for osd.0 2024-04-26T00:24:38.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738374 for osd.1 2024-04-26T00:24:39.393 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:39.393 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:39.698 INFO:teuthology.orchestra.run.smithi072.stdout:34359738375 2024-04-26T00:24:39.700 INFO:teuthology.orchestra.run.smithi072.stdout:34359738375 2024-04-26T00:24:39.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738375 for osd.1 2024-04-26T00:24:39.712 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:39.714 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738375 got 34359738375 for osd.0 2024-04-26T00:24:39.715 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:39.715 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-04-26T00:24:39.715 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:40.033 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:40.034 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:40.044 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-26T00:24:38.398972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54324,"kb_used_data":272,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689356,"statfs":{"total":193273528320,"available":193217900544,"internally_reserved":0,"allocated":278528,"data_stored":72484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.661372"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.541653+0000","last_change":"2024-04-26T00:24:33.522356+0000","last_active":"2024-04-26T00:24:36.541653+0000","last_peered":"2024-04-26T00:24:36.541653+0000","last_clean":"2024-04-26T00:24:36.541653+0000","last_became_active":"2024-04-26T00:24:33.521722+0000","last_became_peered":"2024-04-26T00:24:33.521722+0000","last_unstale":"2024-04-26T00:24:36.541653+0000","last_undegraded":"2024-04-26T00:24:36.541653+0000","last_fullsized":"2024-04-26T00:24:36.541653+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:20.746980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.542074+0000","last_change":"2024-04-26T00:24:35.539017+0000","last_active":"2024-04-26T00:24:32.507794+0000","last_peered":"2024-04-26T00:24:32.507794+0000","last_clean":"2024-04-26T00:24:32.507794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.542074+0000","last_undegraded":"2024-04-26T00:24:36.542074+0000","last_fullsized":"2024-04-26T00:24:36.542074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.537813+0000","last_change":"2024-04-26T00:24:35.537813+0000","last_active":"2024-04-26T00:24:35.537813+0000","last_peered":"2024-04-26T00:24:34.530355+0000","last_clean":"2024-04-26T00:24:34.530355+0000","last_became_active":"2024-04-26T00:24:33.527437+0000","last_became_peered":"2024-04-26T00:24:33.527437+0000","last_unstale":"2024-04-26T00:24:35.537813+0000","last_undegraded":"2024-04-26T00:24:35.537813+0000","last_fullsized":"2024-04-26T00:24:35.537813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.542445+0000","last_change":"2024-04-26T00:24:35.539498+0000","last_active":"2024-04-26T00:24:32.507794+0000","last_peered":"2024-04-26T00:24:32.507794+0000","last_clean":"2024-04-26T00:24:32.507794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.542445+0000","last_undegraded":"2024-04-26T00:24:36.542445+0000","last_fullsized":"2024-04-26T00:24:36.542445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.538914+0000","last_change":"2024-04-26T00:24:35.538914+0000","last_active":"2024-04-26T00:24:35.538914+0000","last_peered":"2024-04-26T00:24:34.530118+0000","last_clean":"2024-04-26T00:24:34.530118+0000","last_became_active":"2024-04-26T00:24:33.526127+0000","last_became_peered":"2024-04-26T00:24:33.526127+0000","last_unstale":"2024-04-26T00:24:35.538914+0000","last_undegraded":"2024-04-26T00:24:35.538914+0000","last_fullsized":"2024-04-26T00:24:35.538914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540264+0000","last_change":"2024-04-26T00:24:33.524091+0000","last_active":"2024-04-26T00:24:36.540264+0000","last_peered":"2024-04-26T00:24:36.540264+0000","last_clean":"2024-04-26T00:24:36.540264+0000","last_became_active":"2024-04-26T00:24:33.523813+0000","last_became_peered":"2024-04-26T00:24:33.523813+0000","last_unstale":"2024-04-26T00:24:36.540264+0000","last_undegraded":"2024-04-26T00:24:36.540264+0000","last_fullsized":"2024-04-26T00:24:36.540264+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:17:35.604631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540054+0000","last_change":"2024-04-26T00:24:33.523560+0000","last_active":"2024-04-26T00:24:36.540054+0000","last_peered":"2024-04-26T00:24:36.540054+0000","last_clean":"2024-04-26T00:24:36.540054+0000","last_became_active":"2024-04-26T00:24:33.523263+0000","last_became_peered":"2024-04-26T00:24:33.523263+0000","last_unstale":"2024-04-26T00:24:36.540054+0000","last_undegraded":"2024-04-26T00:24:36.540054+0000","last_fullsized":"2024-04-26T00:24:36.540054+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:54.043007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.537702+0000","last_change":"2024-04-26T00:24:35.537702+0000","last_active":"2024-04-26T00:24:35.537702+0000","last_peered":"2024-04-26T00:24:34.529933+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:24:33.525386+0000","last_became_peered":"2024-04-26T00:24:33.525386+0000","last_unstale":"2024-04-26T00:24:35.537702+0000","last_undegraded":"2024-04-26T00:24:35.537702+0000","last_fullsized":"2024-04-26T00:24:35.537702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540489+0000","last_change":"2024-04-26T00:24:24.484927+0000","last_active":"2024-04-26T00:24:36.540489+0000","last_peered":"2024-04-26T00:24:36.540489+0000","last_clean":"2024-04-26T00:24:36.540489+0000","last_became_active":"2024-04-26T00:24:24.484616+0000","last_became_peered":"2024-04-26T00:24:24.484616+0000","last_unstale":"2024-04-26T00:24:36.540489+0000","last_undegraded":"2024-04-26T00:24:36.540489+0000","last_fullsized":"2024-04-26T00:24:36.540489+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:37.127557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.537175+0000","last_change":"2024-04-26T00:24:35.537175+0000","last_active":"2024-04-26T00:24:35.537175+0000","last_peered":"2024-04-26T00:24:34.529658+0000","last_clean":"2024-04-26T00:24:34.529658+0000","last_became_active":"2024-04-26T00:24:33.526846+0000","last_became_peered":"2024-04-26T00:24:33.526846+0000","last_unstale":"2024-04-26T00:24:35.537175+0000","last_undegraded":"2024-04-26T00:24:35.537175+0000","last_fullsized":"2024-04-26T00:24:35.537175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.540511+0000","last_change":"2024-04-26T00:24:35.537001+0000","last_active":"2024-04-26T00:24:23.468323+0000","last_peered":"2024-04-26T00:24:23.468323+0000","last_clean":"2024-04-26T00:24:23.468323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.540511+0000","last_undegraded":"2024-04-26T00:24:36.540511+0000","last_fullsized":"2024-04-26T00:24:36.540511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540533+0000","last_change":"2024-04-26T00:24:33.520958+0000","last_active":"2024-04-26T00:24:36.540533+0000","last_peered":"2024-04-26T00:24:36.540533+0000","last_clean":"2024-04-26T00:24:36.540533+0000","last_became_active":"2024-04-26T00:24:33.520391+0000","last_became_peered":"2024-04-26T00:24:33.520391+0000","last_unstale":"2024-04-26T00:24:36.540533+0000","last_undegraded":"2024-04-26T00:24:36.540533+0000","last_fullsized":"2024-04-26T00:24:36.540533+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:26:12.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":22,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540635+0000","last_change":"2024-04-26T00:24:24.487504+0000","last_active":"2024-04-26T00:24:36.540635+0000","last_peered":"2024-04-26T00:24:36.540635+0000","last_clean":"2024-04-26T00:24:36.540635+0000","last_became_active":"2024-04-26T00:24:24.486992+0000","last_became_peered":"2024-04-26T00:24:24.486992+0000","last_unstale":"2024-04-26T00:24:36.540635+0000","last_undegraded":"2024-04-26T00:24:36.540635+0000","last_fullsized":"2024-04-26T00:24:36.540635+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.541662+0000","last_change":"2024-04-26T00:24:35.537119+0000","last_active":"2024-04-26T00:24:32.507794+0000","last_peered":"2024-04-26T00:24:32.507794+0000","last_clean":"2024-04-26T00:24:32.507794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.541662+0000","last_undegraded":"2024-04-26T00:24:36.541662+0000","last_fullsized":"2024-04-26T00:24:36.541662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.537526+0000","last_change":"2024-04-26T00:24:35.537526+0000","last_active":"2024-04-26T00:24:35.537526+0000","last_peered":"2024-04-26T00:24:34.529750+0000","last_clean":"2024-04-26T00:24:34.529750+0000","last_became_active":"2024-04-26T00:24:24.486618+0000","last_became_peered":"2024-04-26T00:24:24.486618+0000","last_unstale":"2024-04-26T00:24:35.537526+0000","last_undegraded":"2024-04-26T00:24:35.537526+0000","last_fullsized":"2024-04-26T00:24:35.537526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.540302+0000","last_change":"2024-04-26T00:24:35.537001+0000","last_active":"2024-04-26T00:24:32.507794+0000","last_peered":"2024-04-26T00:24:32.507794+0000","last_clean":"2024-04-26T00:24:32.507794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.540302+0000","last_undegraded":"2024-04-26T00:24:36.540302+0000","last_fullsized":"2024-04-26T00:24:36.540302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540957+0000","last_change":"2024-04-26T00:24:33.524944+0000","last_active":"2024-04-26T00:24:36.540957+0000","last_peered":"2024-04-26T00:24:36.540957+0000","last_clean":"2024-04-26T00:24:36.540957+0000","last_became_active":"2024-04-26T00:24:33.524513+0000","last_became_peered":"2024-04-26T00:24:33.524513+0000","last_unstale":"2024-04-26T00:24:36.540957+0000","last_undegraded":"2024-04-26T00:24:36.540957+0000","last_fullsized":"2024-04-26T00:24:36.540957+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:18.648174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540376+0000","last_change":"2024-04-26T00:24:24.484819+0000","last_active":"2024-04-26T00:24:36.540376+0000","last_peered":"2024-04-26T00:24:36.540376+0000","last_clean":"2024-04-26T00:24:36.540376+0000","last_became_active":"2024-04-26T00:24:24.484550+0000","last_became_peered":"2024-04-26T00:24:24.484550+0000","last_unstale":"2024-04-26T00:24:36.540376+0000","last_undegraded":"2024-04-26T00:24:36.540376+0000","last_fullsized":"2024-04-26T00:24:36.540376+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:24.731073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540032+0000","last_change":"2024-04-26T00:24:33.524138+0000","last_active":"2024-04-26T00:24:36.540032+0000","last_peered":"2024-04-26T00:24:36.540032+0000","last_clean":"2024-04-26T00:24:36.540032+0000","last_became_active":"2024-04-26T00:24:33.523888+0000","last_became_peered":"2024-04-26T00:24:33.523888+0000","last_unstale":"2024-04-26T00:24:36.540032+0000","last_undegraded":"2024-04-26T00:24:36.540032+0000","last_fullsized":"2024-04-26T00:24:36.540032+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:08:26.388084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.542052+0000","last_change":"2024-04-26T00:24:35.537669+0000","last_active":"2024-04-26T00:24:23.468323+0000","last_peered":"2024-04-26T00:24:23.468323+0000","last_clean":"2024-04-26T00:24:23.468323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.542052+0000","last_undegraded":"2024-04-26T00:24:36.542052+0000","last_fullsized":"2024-04-26T00:24:36.542052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540757+0000","last_change":"2024-04-26T00:24:33.523842+0000","last_active":"2024-04-26T00:24:36.540757+0000","last_peered":"2024-04-26T00:24:36.540757+0000","last_clean":"2024-04-26T00:24:36.540757+0000","last_became_active":"2024-04-26T00:24:33.523518+0000","last_became_peered":"2024-04-26T00:24:33.523518+0000","last_unstale":"2024-04-26T00:24:36.540757+0000","last_undegraded":"2024-04-26T00:24:36.540757+0000","last_fullsized":"2024-04-26T00:24:36.540757+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:28:10.321141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540153+0000","last_change":"2024-04-26T00:24:24.485307+0000","last_active":"2024-04-26T00:24:36.540153+0000","last_peered":"2024-04-26T00:24:36.540153+0000","last_clean":"2024-04-26T00:24:36.540153+0000","last_became_active":"2024-04-26T00:24:24.484577+0000","last_became_peered":"2024-04-26T00:24:24.484577+0000","last_unstale":"2024-04-26T00:24:36.540153+0000","last_undegraded":"2024-04-26T00:24:36.540153+0000","last_fullsized":"2024-04-26T00:24:36.540153+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:22:47.688275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.538178+0000","last_change":"2024-04-26T00:24:35.538178+0000","last_active":"2024-04-26T00:24:35.538178+0000","last_peered":"2024-04-26T00:24:34.530159+0000","last_clean":"2024-04-26T00:24:34.530159+0000","last_became_active":"2024-04-26T00:24:24.487963+0000","last_became_peered":"2024-04-26T00:24:24.487963+0000","last_unstale":"2024-04-26T00:24:35.538178+0000","last_undegraded":"2024-04-26T00:24:35.538178+0000","last_fullsized":"2024-04-26T00:24:35.538178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540328+0000","last_change":"2024-04-26T00:24:33.524248+0000","last_active":"2024-04-26T00:24:36.540328+0000","last_peered":"2024-04-26T00:24:36.540328+0000","last_clean":"2024-04-26T00:24:36.540328+0000","last_became_active":"2024-04-26T00:24:33.523571+0000","last_became_peered":"2024-04-26T00:24:33.523571+0000","last_unstale":"2024-04-26T00:24:36.540328+0000","last_undegraded":"2024-04-26T00:24:36.540328+0000","last_fullsized":"2024-04-26T00:24:36.540328+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:31.723150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":139264,"data_stored":36242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":139264,"data_stored":36242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:40.046 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:40.323 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:40.324 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:40.335 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2024-04-26T00:24:38.398972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54324,"kb_used_data":272,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689356,"statfs":{"total":193273528320,"available":193217900544,"internally_reserved":0,"allocated":278528,"data_stored":72484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.661372"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.541653+0000","last_change":"2024-04-26T00:24:33.522356+0000","last_active":"2024-04-26T00:24:36.541653+0000","last_peered":"2024-04-26T00:24:36.541653+0000","last_clean":"2024-04-26T00:24:36.541653+0000","last_became_active":"2024-04-26T00:24:33.521722+0000","last_became_peered":"2024-04-26T00:24:33.521722+0000","last_unstale":"2024-04-26T00:24:36.541653+0000","last_undegraded":"2024-04-26T00:24:36.541653+0000","last_fullsized":"2024-04-26T00:24:36.541653+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:20.746980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.542074+0000","last_change":"2024-04-26T00:24:35.539017+0000","last_active":"2024-04-26T00:24:32.507794+0000","last_peered":"2024-04-26T00:24:32.507794+0000","last_clean":"2024-04-26T00:24:32.507794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.542074+0000","last_undegraded":"2024-04-26T00:24:36.542074+0000","last_fullsized":"2024-04-26T00:24:36.542074+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.537813+0000","last_change":"2024-04-26T00:24:35.537813+0000","last_active":"2024-04-26T00:24:35.537813+0000","last_peered":"2024-04-26T00:24:34.530355+0000","last_clean":"2024-04-26T00:24:34.530355+0000","last_became_active":"2024-04-26T00:24:33.527437+0000","last_became_peered":"2024-04-26T00:24:33.527437+0000","last_unstale":"2024-04-26T00:24:35.537813+0000","last_undegraded":"2024-04-26T00:24:35.537813+0000","last_fullsized":"2024-04-26T00:24:35.537813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.542445+0000","last_change":"2024-04-26T00:24:35.539498+0000","last_active":"2024-04-26T00:24:32.507794+0000","last_peered":"2024-04-26T00:24:32.507794+0000","last_clean":"2024-04-26T00:24:32.507794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.542445+0000","last_undegraded":"2024-04-26T00:24:36.542445+0000","last_fullsized":"2024-04-26T00:24:36.542445+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.538914+0000","last_change":"2024-04-26T00:24:35.538914+0000","last_active":"2024-04-26T00:24:35.538914+0000","last_peered":"2024-04-26T00:24:34.530118+0000","last_clean":"2024-04-26T00:24:34.530118+0000","last_became_active":"2024-04-26T00:24:33.526127+0000","last_became_peered":"2024-04-26T00:24:33.526127+0000","last_unstale":"2024-04-26T00:24:35.538914+0000","last_undegraded":"2024-04-26T00:24:35.538914+0000","last_fullsized":"2024-04-26T00:24:35.538914+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540264+0000","last_change":"2024-04-26T00:24:33.524091+0000","last_active":"2024-04-26T00:24:36.540264+0000","last_peered":"2024-04-26T00:24:36.540264+0000","last_clean":"2024-04-26T00:24:36.540264+0000","last_became_active":"2024-04-26T00:24:33.523813+0000","last_became_peered":"2024-04-26T00:24:33.523813+0000","last_unstale":"2024-04-26T00:24:36.540264+0000","last_undegraded":"2024-04-26T00:24:36.540264+0000","last_fullsized":"2024-04-26T00:24:36.540264+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:17:35.604631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540054+0000","last_change":"2024-04-26T00:24:33.523560+0000","last_active":"2024-04-26T00:24:36.540054+0000","last_peered":"2024-04-26T00:24:36.540054+0000","last_clean":"2024-04-26T00:24:36.540054+0000","last_became_active":"2024-04-26T00:24:33.523263+0000","last_became_peered":"2024-04-26T00:24:33.523263+0000","last_unstale":"2024-04-26T00:24:36.540054+0000","last_undegraded":"2024-04-26T00:24:36.540054+0000","last_fullsized":"2024-04-26T00:24:36.540054+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:54.043007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.537702+0000","last_change":"2024-04-26T00:24:35.537702+0000","last_active":"2024-04-26T00:24:35.537702+0000","last_peered":"2024-04-26T00:24:34.529933+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:24:33.525386+0000","last_became_peered":"2024-04-26T00:24:33.525386+0000","last_unstale":"2024-04-26T00:24:35.537702+0000","last_undegraded":"2024-04-26T00:24:35.537702+0000","last_fullsized":"2024-04-26T00:24:35.537702+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540489+0000","last_change":"2024-04-26T00:24:24.484927+0000","last_active":"2024-04-26T00:24:36.540489+0000","last_peered":"2024-04-26T00:24:36.540489+0000","last_clean":"2024-04-26T00:24:36.540489+0000","last_became_active":"2024-04-26T00:24:24.484616+0000","last_became_peered":"2024-04-26T00:24:24.484616+0000","last_unstale":"2024-04-26T00:24:36.540489+0000","last_undegraded":"2024-04-26T00:24:36.540489+0000","last_fullsized":"2024-04-26T00:24:36.540489+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:37.127557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.537175+0000","last_change":"2024-04-26T00:24:35.537175+0000","last_active":"2024-04-26T00:24:35.537175+0000","last_peered":"2024-04-26T00:24:34.529658+0000","last_clean":"2024-04-26T00:24:34.529658+0000","last_became_active":"2024-04-26T00:24:33.526846+0000","last_became_peered":"2024-04-26T00:24:33.526846+0000","last_unstale":"2024-04-26T00:24:35.537175+0000","last_undegraded":"2024-04-26T00:24:35.537175+0000","last_fullsized":"2024-04-26T00:24:35.537175+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.540511+0000","last_change":"2024-04-26T00:24:35.537001+0000","last_active":"2024-04-26T00:24:23.468323+0000","last_peered":"2024-04-26T00:24:23.468323+0000","last_clean":"2024-04-26T00:24:23.468323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.540511+0000","last_undegraded":"2024-04-26T00:24:36.540511+0000","last_fullsized":"2024-04-26T00:24:36.540511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540533+0000","last_change":"2024-04-26T00:24:33.520958+0000","last_active":"2024-04-26T00:24:36.540533+0000","last_peered":"2024-04-26T00:24:36.540533+0000","last_clean":"2024-04-26T00:24:36.540533+0000","last_became_active":"2024-04-26T00:24:33.520391+0000","last_became_peered":"2024-04-26T00:24:33.520391+0000","last_unstale":"2024-04-26T00:24:36.540533+0000","last_undegraded":"2024-04-26T00:24:36.540533+0000","last_fullsized":"2024-04-26T00:24:36.540533+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:26:12.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":22,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540635+0000","last_change":"2024-04-26T00:24:24.487504+0000","last_active":"2024-04-26T00:24:36.540635+0000","last_peered":"2024-04-26T00:24:36.540635+0000","last_clean":"2024-04-26T00:24:36.540635+0000","last_became_active":"2024-04-26T00:24:24.486992+0000","last_became_peered":"2024-04-26T00:24:24.486992+0000","last_unstale":"2024-04-26T00:24:36.540635+0000","last_undegraded":"2024-04-26T00:24:36.540635+0000","last_fullsized":"2024-04-26T00:24:36.540635+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.541662+0000","last_change":"2024-04-26T00:24:35.537119+0000","last_active":"2024-04-26T00:24:32.507794+0000","last_peered":"2024-04-26T00:24:32.507794+0000","last_clean":"2024-04-26T00:24:32.507794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.541662+0000","last_undegraded":"2024-04-26T00:24:36.541662+0000","last_fullsized":"2024-04-26T00:24:36.541662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.537526+0000","last_change":"2024-04-26T00:24:35.537526+0000","last_active":"2024-04-26T00:24:35.537526+0000","last_peered":"2024-04-26T00:24:34.529750+0000","last_clean":"2024-04-26T00:24:34.529750+0000","last_became_active":"2024-04-26T00:24:24.486618+0000","last_became_peered":"2024-04-26T00:24:24.486618+0000","last_unstale":"2024-04-26T00:24:35.537526+0000","last_undegraded":"2024-04-26T00:24:35.537526+0000","last_fullsized":"2024-04-26T00:24:35.537526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.540302+0000","last_change":"2024-04-26T00:24:35.537001+0000","last_active":"2024-04-26T00:24:32.507794+0000","last_peered":"2024-04-26T00:24:32.507794+0000","last_clean":"2024-04-26T00:24:32.507794+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.540302+0000","last_undegraded":"2024-04-26T00:24:36.540302+0000","last_fullsized":"2024-04-26T00:24:36.540302+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540957+0000","last_change":"2024-04-26T00:24:33.524944+0000","last_active":"2024-04-26T00:24:36.540957+0000","last_peered":"2024-04-26T00:24:36.540957+0000","last_clean":"2024-04-26T00:24:36.540957+0000","last_became_active":"2024-04-26T00:24:33.524513+0000","last_became_peered":"2024-04-26T00:24:33.524513+0000","last_unstale":"2024-04-26T00:24:36.540957+0000","last_undegraded":"2024-04-26T00:24:36.540957+0000","last_fullsized":"2024-04-26T00:24:36.540957+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:18.648174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540376+0000","last_change":"2024-04-26T00:24:24.484819+0000","last_active":"2024-04-26T00:24:36.540376+0000","last_peered":"2024-04-26T00:24:36.540376+0000","last_clean":"2024-04-26T00:24:36.540376+0000","last_became_active":"2024-04-26T00:24:24.484550+0000","last_became_peered":"2024-04-26T00:24:24.484550+0000","last_unstale":"2024-04-26T00:24:36.540376+0000","last_undegraded":"2024-04-26T00:24:36.540376+0000","last_fullsized":"2024-04-26T00:24:36.540376+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:24.731073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540032+0000","last_change":"2024-04-26T00:24:33.524138+0000","last_active":"2024-04-26T00:24:36.540032+0000","last_peered":"2024-04-26T00:24:36.540032+0000","last_clean":"2024-04-26T00:24:36.540032+0000","last_became_active":"2024-04-26T00:24:33.523888+0000","last_became_peered":"2024-04-26T00:24:33.523888+0000","last_unstale":"2024-04-26T00:24:36.540032+0000","last_undegraded":"2024-04-26T00:24:36.540032+0000","last_fullsized":"2024-04-26T00:24:36.540032+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:08:26.388084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":17,"state":"peering","last_fresh":"2024-04-26T00:24:36.542052+0000","last_change":"2024-04-26T00:24:35.537669+0000","last_active":"2024-04-26T00:24:23.468323+0000","last_peered":"2024-04-26T00:24:23.468323+0000","last_clean":"2024-04-26T00:24:23.468323+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-26T00:24:36.542052+0000","last_undegraded":"2024-04-26T00:24:36.542052+0000","last_fullsized":"2024-04-26T00:24:36.542052+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540757+0000","last_change":"2024-04-26T00:24:33.523842+0000","last_active":"2024-04-26T00:24:36.540757+0000","last_peered":"2024-04-26T00:24:36.540757+0000","last_clean":"2024-04-26T00:24:36.540757+0000","last_became_active":"2024-04-26T00:24:33.523518+0000","last_became_peered":"2024-04-26T00:24:33.523518+0000","last_unstale":"2024-04-26T00:24:36.540757+0000","last_undegraded":"2024-04-26T00:24:36.540757+0000","last_fullsized":"2024-04-26T00:24:36.540757+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:28:10.321141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540153+0000","last_change":"2024-04-26T00:24:24.485307+0000","last_active":"2024-04-26T00:24:36.540153+0000","last_peered":"2024-04-26T00:24:36.540153+0000","last_clean":"2024-04-26T00:24:36.540153+0000","last_became_active":"2024-04-26T00:24:24.484577+0000","last_became_peered":"2024-04-26T00:24:24.484577+0000","last_unstale":"2024-04-26T00:24:36.540153+0000","last_undegraded":"2024-04-26T00:24:36.540153+0000","last_fullsized":"2024-04-26T00:24:36.540153+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:22:47.688275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":16,"state":"peering","last_fresh":"2024-04-26T00:24:35.538178+0000","last_change":"2024-04-26T00:24:35.538178+0000","last_active":"2024-04-26T00:24:35.538178+0000","last_peered":"2024-04-26T00:24:34.530159+0000","last_clean":"2024-04-26T00:24:34.530159+0000","last_became_active":"2024-04-26T00:24:24.487963+0000","last_became_peered":"2024-04-26T00:24:24.487963+0000","last_unstale":"2024-04-26T00:24:35.538178+0000","last_undegraded":"2024-04-26T00:24:35.538178+0000","last_fullsized":"2024-04-26T00:24:35.538178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-26T00:24:36.540328+0000","last_change":"2024-04-26T00:24:33.524248+0000","last_active":"2024-04-26T00:24:36.540328+0000","last_peered":"2024-04-26T00:24:36.540328+0000","last_clean":"2024-04-26T00:24:36.540328+0000","last_became_active":"2024-04-26T00:24:33.523571+0000","last_became_peered":"2024-04-26T00:24:33.523571+0000","last_unstale":"2024-04-26T00:24:36.540328+0000","last_undegraded":"2024-04-26T00:24:36.540328+0000","last_fullsized":"2024-04-26T00:24:36.540328+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:31.723150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":139264,"data_stored":36242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738375,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27160,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344680,"statfs":{"total":96636764160,"available":96608952320,"internally_reserved":0,"allocated":139264,"data_stored":36242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:40.337 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:40.614 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:40.615 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:40.626 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2024-04-26T00:24:40.399225+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54356,"kb_used_data":312,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689324,"statfs":{"total":193273528320,"available":193217867776,"internally_reserved":0,"allocated":319488,"data_stored":84434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.661625"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.741550+0000","last_change":"2024-04-26T00:24:33.522356+0000","last_active":"2024-04-26T00:24:37.741550+0000","last_peered":"2024-04-26T00:24:37.741550+0000","last_clean":"2024-04-26T00:24:37.741550+0000","last_became_active":"2024-04-26T00:24:33.521722+0000","last_became_peered":"2024-04-26T00:24:33.521722+0000","last_unstale":"2024-04-26T00:24:37.741550+0000","last_undegraded":"2024-04-26T00:24:37.741550+0000","last_fullsized":"2024-04-26T00:24:37.741550+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:20.746980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.746587+0000","last_change":"2024-04-26T00:24:37.746587+0000","last_active":"2024-04-26T00:24:37.746587+0000","last_peered":"2024-04-26T00:24:37.746587+0000","last_clean":"2024-04-26T00:24:37.746587+0000","last_became_active":"2024-04-26T00:24:37.746244+0000","last_became_peered":"2024-04-26T00:24:37.746244+0000","last_unstale":"2024-04-26T00:24:37.746587+0000","last_undegraded":"2024-04-26T00:24:37.746587+0000","last_fullsized":"2024-04-26T00:24:37.746587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:58:33.836267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744089+0000","last_change":"2024-04-26T00:24:37.744089+0000","last_active":"2024-04-26T00:24:37.744089+0000","last_peered":"2024-04-26T00:24:37.744089+0000","last_clean":"2024-04-26T00:24:37.744089+0000","last_became_active":"2024-04-26T00:24:37.743758+0000","last_became_peered":"2024-04-26T00:24:37.743758+0000","last_unstale":"2024-04-26T00:24:37.744089+0000","last_undegraded":"2024-04-26T00:24:37.744089+0000","last_fullsized":"2024-04-26T00:24:37.744089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:40.569589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.745237+0000","last_change":"2024-04-26T00:24:37.745237+0000","last_active":"2024-04-26T00:24:37.745237+0000","last_peered":"2024-04-26T00:24:37.745237+0000","last_clean":"2024-04-26T00:24:37.745237+0000","last_became_active":"2024-04-26T00:24:37.744793+0000","last_became_peered":"2024-04-26T00:24:37.744793+0000","last_unstale":"2024-04-26T00:24:37.745237+0000","last_undegraded":"2024-04-26T00:24:37.745237+0000","last_fullsized":"2024-04-26T00:24:37.745237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:40:23.343283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744784+0000","last_change":"2024-04-26T00:24:37.744784+0000","last_active":"2024-04-26T00:24:37.744784+0000","last_peered":"2024-04-26T00:24:37.744784+0000","last_clean":"2024-04-26T00:24:37.744784+0000","last_became_active":"2024-04-26T00:24:37.744384+0000","last_became_peered":"2024-04-26T00:24:37.744384+0000","last_unstale":"2024-04-26T00:24:37.744784+0000","last_undegraded":"2024-04-26T00:24:37.744784+0000","last_fullsized":"2024-04-26T00:24:37.744784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:28:59.172280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738640+0000","last_change":"2024-04-26T00:24:33.524091+0000","last_active":"2024-04-26T00:24:37.738640+0000","last_peered":"2024-04-26T00:24:37.738640+0000","last_clean":"2024-04-26T00:24:37.738640+0000","last_became_active":"2024-04-26T00:24:33.523813+0000","last_became_peered":"2024-04-26T00:24:33.523813+0000","last_unstale":"2024-04-26T00:24:37.738640+0000","last_undegraded":"2024-04-26T00:24:37.738640+0000","last_fullsized":"2024-04-26T00:24:37.738640+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:17:35.604631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738707+0000","last_change":"2024-04-26T00:24:33.523560+0000","last_active":"2024-04-26T00:24:37.738707+0000","last_peered":"2024-04-26T00:24:37.738707+0000","last_clean":"2024-04-26T00:24:37.738707+0000","last_became_active":"2024-04-26T00:24:33.523263+0000","last_became_peered":"2024-04-26T00:24:33.523263+0000","last_unstale":"2024-04-26T00:24:37.738707+0000","last_undegraded":"2024-04-26T00:24:37.738707+0000","last_fullsized":"2024-04-26T00:24:37.738707+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:54.043007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.746821+0000","last_change":"2024-04-26T00:24:37.746821+0000","last_active":"2024-04-26T00:24:37.746821+0000","last_peered":"2024-04-26T00:24:37.746821+0000","last_clean":"2024-04-26T00:24:37.746821+0000","last_became_active":"2024-04-26T00:24:37.746432+0000","last_became_peered":"2024-04-26T00:24:37.746432+0000","last_unstale":"2024-04-26T00:24:37.746821+0000","last_undegraded":"2024-04-26T00:24:37.746821+0000","last_fullsized":"2024-04-26T00:24:37.746821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:15:52.618042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738411+0000","last_change":"2024-04-26T00:24:24.484927+0000","last_active":"2024-04-26T00:24:37.738411+0000","last_peered":"2024-04-26T00:24:37.738411+0000","last_clean":"2024-04-26T00:24:37.738411+0000","last_became_active":"2024-04-26T00:24:24.484616+0000","last_became_peered":"2024-04-26T00:24:24.484616+0000","last_unstale":"2024-04-26T00:24:37.738411+0000","last_undegraded":"2024-04-26T00:24:37.738411+0000","last_fullsized":"2024-04-26T00:24:37.738411+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:37.127557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.745601+0000","last_change":"2024-04-26T00:24:37.745601+0000","last_active":"2024-04-26T00:24:37.745601+0000","last_peered":"2024-04-26T00:24:37.745601+0000","last_clean":"2024-04-26T00:24:37.745601+0000","last_became_active":"2024-04-26T00:24:37.745109+0000","last_became_peered":"2024-04-26T00:24:37.745109+0000","last_unstale":"2024-04-26T00:24:37.745601+0000","last_undegraded":"2024-04-26T00:24:37.745601+0000","last_fullsized":"2024-04-26T00:24:37.745601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:04.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744664+0000","last_change":"2024-04-26T00:24:37.744664+0000","last_active":"2024-04-26T00:24:37.744664+0000","last_peered":"2024-04-26T00:24:37.744664+0000","last_clean":"2024-04-26T00:24:37.744664+0000","last_became_active":"2024-04-26T00:24:37.744351+0000","last_became_peered":"2024-04-26T00:24:37.744351+0000","last_unstale":"2024-04-26T00:24:37.744664+0000","last_undegraded":"2024-04-26T00:24:37.744664+0000","last_fullsized":"2024-04-26T00:24:37.744664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:57.988445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.741079+0000","last_change":"2024-04-26T00:24:33.520958+0000","last_active":"2024-04-26T00:24:37.741079+0000","last_peered":"2024-04-26T00:24:37.741079+0000","last_clean":"2024-04-26T00:24:37.741079+0000","last_became_active":"2024-04-26T00:24:33.520391+0000","last_became_peered":"2024-04-26T00:24:33.520391+0000","last_unstale":"2024-04-26T00:24:37.741079+0000","last_undegraded":"2024-04-26T00:24:37.741079+0000","last_fullsized":"2024-04-26T00:24:37.741079+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:26:12.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.741146+0000","last_change":"2024-04-26T00:24:24.487504+0000","last_active":"2024-04-26T00:24:37.741146+0000","last_peered":"2024-04-26T00:24:37.741146+0000","last_clean":"2024-04-26T00:24:37.741146+0000","last_became_active":"2024-04-26T00:24:24.486992+0000","last_became_peered":"2024-04-26T00:24:24.486992+0000","last_unstale":"2024-04-26T00:24:37.741146+0000","last_undegraded":"2024-04-26T00:24:37.741146+0000","last_fullsized":"2024-04-26T00:24:37.741146+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744864+0000","last_change":"2024-04-26T00:24:37.744864+0000","last_active":"2024-04-26T00:24:37.744864+0000","last_peered":"2024-04-26T00:24:37.744864+0000","last_clean":"2024-04-26T00:24:37.744864+0000","last_became_active":"2024-04-26T00:24:37.744531+0000","last_became_peered":"2024-04-26T00:24:37.744531+0000","last_unstale":"2024-04-26T00:24:37.744864+0000","last_undegraded":"2024-04-26T00:24:37.744864+0000","last_fullsized":"2024-04-26T00:24:37.744864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:46.185834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.745730+0000","last_change":"2024-04-26T00:24:37.745730+0000","last_active":"2024-04-26T00:24:37.745730+0000","last_peered":"2024-04-26T00:24:37.745730+0000","last_clean":"2024-04-26T00:24:37.745730+0000","last_became_active":"2024-04-26T00:24:37.745192+0000","last_became_peered":"2024-04-26T00:24:37.745192+0000","last_unstale":"2024-04-26T00:24:37.745730+0000","last_undegraded":"2024-04-26T00:24:37.745730+0000","last_fullsized":"2024-04-26T00:24:37.745730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:55.686116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.745797+0000","last_change":"2024-04-26T00:24:37.745797+0000","last_active":"2024-04-26T00:24:37.745797+0000","last_peered":"2024-04-26T00:24:37.745797+0000","last_clean":"2024-04-26T00:24:37.745797+0000","last_became_active":"2024-04-26T00:24:37.745440+0000","last_became_peered":"2024-04-26T00:24:37.745440+0000","last_unstale":"2024-04-26T00:24:37.745797+0000","last_undegraded":"2024-04-26T00:24:37.745797+0000","last_fullsized":"2024-04-26T00:24:37.745797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:47:50.091953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.739400+0000","last_change":"2024-04-26T00:24:33.524944+0000","last_active":"2024-04-26T00:24:37.739400+0000","last_peered":"2024-04-26T00:24:37.739400+0000","last_clean":"2024-04-26T00:24:37.739400+0000","last_became_active":"2024-04-26T00:24:33.524513+0000","last_became_peered":"2024-04-26T00:24:33.524513+0000","last_unstale":"2024-04-26T00:24:37.739400+0000","last_undegraded":"2024-04-26T00:24:37.739400+0000","last_fullsized":"2024-04-26T00:24:37.739400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:18.648174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.739204+0000","last_change":"2024-04-26T00:24:24.484819+0000","last_active":"2024-04-26T00:24:37.739204+0000","last_peered":"2024-04-26T00:24:37.739204+0000","last_clean":"2024-04-26T00:24:37.739204+0000","last_became_active":"2024-04-26T00:24:24.484550+0000","last_became_peered":"2024-04-26T00:24:24.484550+0000","last_unstale":"2024-04-26T00:24:37.739204+0000","last_undegraded":"2024-04-26T00:24:37.739204+0000","last_fullsized":"2024-04-26T00:24:37.739204+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:24.731073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738789+0000","last_change":"2024-04-26T00:24:33.524138+0000","last_active":"2024-04-26T00:24:37.738789+0000","last_peered":"2024-04-26T00:24:37.738789+0000","last_clean":"2024-04-26T00:24:37.738789+0000","last_became_active":"2024-04-26T00:24:33.523888+0000","last_became_peered":"2024-04-26T00:24:33.523888+0000","last_unstale":"2024-04-26T00:24:37.738789+0000","last_undegraded":"2024-04-26T00:24:37.738789+0000","last_fullsized":"2024-04-26T00:24:37.738789+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:08:26.388084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.746756+0000","last_change":"2024-04-26T00:24:37.746756+0000","last_active":"2024-04-26T00:24:37.746756+0000","last_peered":"2024-04-26T00:24:37.746756+0000","last_clean":"2024-04-26T00:24:37.746756+0000","last_became_active":"2024-04-26T00:24:37.746330+0000","last_became_peered":"2024-04-26T00:24:37.746330+0000","last_unstale":"2024-04-26T00:24:37.746756+0000","last_undegraded":"2024-04-26T00:24:37.746756+0000","last_fullsized":"2024-04-26T00:24:37.746756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:49:00.432434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.739226+0000","last_change":"2024-04-26T00:24:33.523842+0000","last_active":"2024-04-26T00:24:37.739226+0000","last_peered":"2024-04-26T00:24:37.739226+0000","last_clean":"2024-04-26T00:24:37.739226+0000","last_became_active":"2024-04-26T00:24:33.523518+0000","last_became_peered":"2024-04-26T00:24:33.523518+0000","last_unstale":"2024-04-26T00:24:37.739226+0000","last_undegraded":"2024-04-26T00:24:37.739226+0000","last_fullsized":"2024-04-26T00:24:37.739226+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:28:10.321141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738848+0000","last_change":"2024-04-26T00:24:24.485307+0000","last_active":"2024-04-26T00:24:37.738848+0000","last_peered":"2024-04-26T00:24:37.738848+0000","last_clean":"2024-04-26T00:24:37.738848+0000","last_became_active":"2024-04-26T00:24:24.484577+0000","last_became_peered":"2024-04-26T00:24:24.484577+0000","last_unstale":"2024-04-26T00:24:37.738848+0000","last_undegraded":"2024-04-26T00:24:37.738848+0000","last_fullsized":"2024-04-26T00:24:37.738848+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:22:47.688275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744120+0000","last_change":"2024-04-26T00:24:37.744120+0000","last_active":"2024-04-26T00:24:37.744120+0000","last_peered":"2024-04-26T00:24:37.744120+0000","last_clean":"2024-04-26T00:24:37.744120+0000","last_became_active":"2024-04-26T00:24:37.743699+0000","last_became_peered":"2024-04-26T00:24:37.743699+0000","last_unstale":"2024-04-26T00:24:37.744120+0000","last_undegraded":"2024-04-26T00:24:37.744120+0000","last_fullsized":"2024-04-26T00:24:37.744120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:28.802989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.739310+0000","last_change":"2024-04-26T00:24:33.524248+0000","last_active":"2024-04-26T00:24:37.739310+0000","last_peered":"2024-04-26T00:24:37.739310+0000","last_clean":"2024-04-26T00:24:37.739310+0000","last_became_active":"2024-04-26T00:24:33.523571+0000","last_became_peered":"2024-04-26T00:24:33.523571+0000","last_unstale":"2024-04-26T00:24:37.739310+0000","last_undegraded":"2024-04-26T00:24:37.739310+0000","last_fullsized":"2024-04-26T00:24:37.739310+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:31.723150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":159744,"data_stored":42217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":42217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:43.628 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:43.918 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:43.918 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:43.930 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-26T00:24:42.399566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54356,"kb_used_data":312,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689324,"statfs":{"total":193273528320,"available":193217867776,"internally_reserved":0,"allocated":319488,"data_stored":84434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.661966"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.741550+0000","last_change":"2024-04-26T00:24:33.522356+0000","last_active":"2024-04-26T00:24:37.741550+0000","last_peered":"2024-04-26T00:24:37.741550+0000","last_clean":"2024-04-26T00:24:37.741550+0000","last_became_active":"2024-04-26T00:24:33.521722+0000","last_became_peered":"2024-04-26T00:24:33.521722+0000","last_unstale":"2024-04-26T00:24:37.741550+0000","last_undegraded":"2024-04-26T00:24:37.741550+0000","last_fullsized":"2024-04-26T00:24:37.741550+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:59:20.746980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.746587+0000","last_change":"2024-04-26T00:24:37.746587+0000","last_active":"2024-04-26T00:24:37.746587+0000","last_peered":"2024-04-26T00:24:37.746587+0000","last_clean":"2024-04-26T00:24:37.746587+0000","last_became_active":"2024-04-26T00:24:37.746244+0000","last_became_peered":"2024-04-26T00:24:37.746244+0000","last_unstale":"2024-04-26T00:24:37.746587+0000","last_undegraded":"2024-04-26T00:24:37.746587+0000","last_fullsized":"2024-04-26T00:24:37.746587+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:58:33.836267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744089+0000","last_change":"2024-04-26T00:24:37.744089+0000","last_active":"2024-04-26T00:24:37.744089+0000","last_peered":"2024-04-26T00:24:37.744089+0000","last_clean":"2024-04-26T00:24:37.744089+0000","last_became_active":"2024-04-26T00:24:37.743758+0000","last_became_peered":"2024-04-26T00:24:37.743758+0000","last_unstale":"2024-04-26T00:24:37.744089+0000","last_undegraded":"2024-04-26T00:24:37.744089+0000","last_fullsized":"2024-04-26T00:24:37.744089+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:09:40.569589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.745237+0000","last_change":"2024-04-26T00:24:37.745237+0000","last_active":"2024-04-26T00:24:37.745237+0000","last_peered":"2024-04-26T00:24:37.745237+0000","last_clean":"2024-04-26T00:24:37.745237+0000","last_became_active":"2024-04-26T00:24:37.744793+0000","last_became_peered":"2024-04-26T00:24:37.744793+0000","last_unstale":"2024-04-26T00:24:37.745237+0000","last_undegraded":"2024-04-26T00:24:37.745237+0000","last_fullsized":"2024-04-26T00:24:37.745237+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:40:23.343283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744784+0000","last_change":"2024-04-26T00:24:37.744784+0000","last_active":"2024-04-26T00:24:37.744784+0000","last_peered":"2024-04-26T00:24:37.744784+0000","last_clean":"2024-04-26T00:24:37.744784+0000","last_became_active":"2024-04-26T00:24:37.744384+0000","last_became_peered":"2024-04-26T00:24:37.744384+0000","last_unstale":"2024-04-26T00:24:37.744784+0000","last_undegraded":"2024-04-26T00:24:37.744784+0000","last_fullsized":"2024-04-26T00:24:37.744784+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:28:59.172280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738640+0000","last_change":"2024-04-26T00:24:33.524091+0000","last_active":"2024-04-26T00:24:37.738640+0000","last_peered":"2024-04-26T00:24:37.738640+0000","last_clean":"2024-04-26T00:24:37.738640+0000","last_became_active":"2024-04-26T00:24:33.523813+0000","last_became_peered":"2024-04-26T00:24:33.523813+0000","last_unstale":"2024-04-26T00:24:37.738640+0000","last_undegraded":"2024-04-26T00:24:37.738640+0000","last_fullsized":"2024-04-26T00:24:37.738640+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:17:35.604631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738707+0000","last_change":"2024-04-26T00:24:33.523560+0000","last_active":"2024-04-26T00:24:37.738707+0000","last_peered":"2024-04-26T00:24:37.738707+0000","last_clean":"2024-04-26T00:24:37.738707+0000","last_became_active":"2024-04-26T00:24:33.523263+0000","last_became_peered":"2024-04-26T00:24:33.523263+0000","last_unstale":"2024-04-26T00:24:37.738707+0000","last_undegraded":"2024-04-26T00:24:37.738707+0000","last_fullsized":"2024-04-26T00:24:37.738707+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:33:54.043007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.746821+0000","last_change":"2024-04-26T00:24:37.746821+0000","last_active":"2024-04-26T00:24:37.746821+0000","last_peered":"2024-04-26T00:24:37.746821+0000","last_clean":"2024-04-26T00:24:37.746821+0000","last_became_active":"2024-04-26T00:24:37.746432+0000","last_became_peered":"2024-04-26T00:24:37.746432+0000","last_unstale":"2024-04-26T00:24:37.746821+0000","last_undegraded":"2024-04-26T00:24:37.746821+0000","last_fullsized":"2024-04-26T00:24:37.746821+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:15:52.618042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738411+0000","last_change":"2024-04-26T00:24:24.484927+0000","last_active":"2024-04-26T00:24:37.738411+0000","last_peered":"2024-04-26T00:24:37.738411+0000","last_clean":"2024-04-26T00:24:37.738411+0000","last_became_active":"2024-04-26T00:24:24.484616+0000","last_became_peered":"2024-04-26T00:24:24.484616+0000","last_unstale":"2024-04-26T00:24:37.738411+0000","last_undegraded":"2024-04-26T00:24:37.738411+0000","last_fullsized":"2024-04-26T00:24:37.738411+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:37.127557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.745601+0000","last_change":"2024-04-26T00:24:37.745601+0000","last_active":"2024-04-26T00:24:37.745601+0000","last_peered":"2024-04-26T00:24:37.745601+0000","last_clean":"2024-04-26T00:24:37.745601+0000","last_became_active":"2024-04-26T00:24:37.745109+0000","last_became_peered":"2024-04-26T00:24:37.745109+0000","last_unstale":"2024-04-26T00:24:37.745601+0000","last_undegraded":"2024-04-26T00:24:37.745601+0000","last_fullsized":"2024-04-26T00:24:37.745601+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:19:04.896584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744664+0000","last_change":"2024-04-26T00:24:37.744664+0000","last_active":"2024-04-26T00:24:37.744664+0000","last_peered":"2024-04-26T00:24:37.744664+0000","last_clean":"2024-04-26T00:24:37.744664+0000","last_became_active":"2024-04-26T00:24:37.744351+0000","last_became_peered":"2024-04-26T00:24:37.744351+0000","last_unstale":"2024-04-26T00:24:37.744664+0000","last_undegraded":"2024-04-26T00:24:37.744664+0000","last_fullsized":"2024-04-26T00:24:37.744664+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:02:57.988445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.741079+0000","last_change":"2024-04-26T00:24:33.520958+0000","last_active":"2024-04-26T00:24:37.741079+0000","last_peered":"2024-04-26T00:24:37.741079+0000","last_clean":"2024-04-26T00:24:37.741079+0000","last_became_active":"2024-04-26T00:24:33.520391+0000","last_became_peered":"2024-04-26T00:24:33.520391+0000","last_unstale":"2024-04-26T00:24:37.741079+0000","last_undegraded":"2024-04-26T00:24:37.741079+0000","last_fullsized":"2024-04-26T00:24:37.741079+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:26:12.637901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.741146+0000","last_change":"2024-04-26T00:24:24.487504+0000","last_active":"2024-04-26T00:24:37.741146+0000","last_peered":"2024-04-26T00:24:37.741146+0000","last_clean":"2024-04-26T00:24:37.741146+0000","last_became_active":"2024-04-26T00:24:24.486992+0000","last_became_peered":"2024-04-26T00:24:24.486992+0000","last_unstale":"2024-04-26T00:24:37.741146+0000","last_undegraded":"2024-04-26T00:24:37.741146+0000","last_fullsized":"2024-04-26T00:24:37.741146+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:57:04.889461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744864+0000","last_change":"2024-04-26T00:24:37.744864+0000","last_active":"2024-04-26T00:24:37.744864+0000","last_peered":"2024-04-26T00:24:37.744864+0000","last_clean":"2024-04-26T00:24:37.744864+0000","last_became_active":"2024-04-26T00:24:37.744531+0000","last_became_peered":"2024-04-26T00:24:37.744531+0000","last_unstale":"2024-04-26T00:24:37.744864+0000","last_undegraded":"2024-04-26T00:24:37.744864+0000","last_fullsized":"2024-04-26T00:24:37.744864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:04:46.185834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.745730+0000","last_change":"2024-04-26T00:24:37.745730+0000","last_active":"2024-04-26T00:24:37.745730+0000","last_peered":"2024-04-26T00:24:37.745730+0000","last_clean":"2024-04-26T00:24:37.745730+0000","last_became_active":"2024-04-26T00:24:37.745192+0000","last_became_peered":"2024-04-26T00:24:37.745192+0000","last_unstale":"2024-04-26T00:24:37.745730+0000","last_undegraded":"2024-04-26T00:24:37.745730+0000","last_fullsized":"2024-04-26T00:24:37.745730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:31:55.686116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.745797+0000","last_change":"2024-04-26T00:24:37.745797+0000","last_active":"2024-04-26T00:24:37.745797+0000","last_peered":"2024-04-26T00:24:37.745797+0000","last_clean":"2024-04-26T00:24:37.745797+0000","last_became_active":"2024-04-26T00:24:37.745440+0000","last_became_peered":"2024-04-26T00:24:37.745440+0000","last_unstale":"2024-04-26T00:24:37.745797+0000","last_undegraded":"2024-04-26T00:24:37.745797+0000","last_fullsized":"2024-04-26T00:24:37.745797+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:47:50.091953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.739400+0000","last_change":"2024-04-26T00:24:33.524944+0000","last_active":"2024-04-26T00:24:37.739400+0000","last_peered":"2024-04-26T00:24:37.739400+0000","last_clean":"2024-04-26T00:24:37.739400+0000","last_became_active":"2024-04-26T00:24:33.524513+0000","last_became_peered":"2024-04-26T00:24:33.524513+0000","last_unstale":"2024-04-26T00:24:37.739400+0000","last_undegraded":"2024-04-26T00:24:37.739400+0000","last_fullsized":"2024-04-26T00:24:37.739400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:54:18.648174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.739204+0000","last_change":"2024-04-26T00:24:24.484819+0000","last_active":"2024-04-26T00:24:37.739204+0000","last_peered":"2024-04-26T00:24:37.739204+0000","last_clean":"2024-04-26T00:24:37.739204+0000","last_became_active":"2024-04-26T00:24:24.484550+0000","last_became_peered":"2024-04-26T00:24:24.484550+0000","last_unstale":"2024-04-26T00:24:37.739204+0000","last_undegraded":"2024-04-26T00:24:37.739204+0000","last_fullsized":"2024-04-26T00:24:37.739204+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:17:24.731073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738789+0000","last_change":"2024-04-26T00:24:33.524138+0000","last_active":"2024-04-26T00:24:37.738789+0000","last_peered":"2024-04-26T00:24:37.738789+0000","last_clean":"2024-04-26T00:24:37.738789+0000","last_became_active":"2024-04-26T00:24:33.523888+0000","last_became_peered":"2024-04-26T00:24:33.523888+0000","last_unstale":"2024-04-26T00:24:37.738789+0000","last_undegraded":"2024-04-26T00:24:37.738789+0000","last_fullsized":"2024-04-26T00:24:37.738789+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:08:26.388084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.746756+0000","last_change":"2024-04-26T00:24:37.746756+0000","last_active":"2024-04-26T00:24:37.746756+0000","last_peered":"2024-04-26T00:24:37.746756+0000","last_clean":"2024-04-26T00:24:37.746756+0000","last_became_active":"2024-04-26T00:24:37.746330+0000","last_became_peered":"2024-04-26T00:24:37.746330+0000","last_unstale":"2024-04-26T00:24:37.746756+0000","last_undegraded":"2024-04-26T00:24:37.746756+0000","last_fullsized":"2024-04-26T00:24:37.746756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:49:00.432434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.739226+0000","last_change":"2024-04-26T00:24:33.523842+0000","last_active":"2024-04-26T00:24:37.739226+0000","last_peered":"2024-04-26T00:24:37.739226+0000","last_clean":"2024-04-26T00:24:37.739226+0000","last_became_active":"2024-04-26T00:24:33.523518+0000","last_became_peered":"2024-04-26T00:24:33.523518+0000","last_unstale":"2024-04-26T00:24:37.739226+0000","last_undegraded":"2024-04-26T00:24:37.739226+0000","last_fullsized":"2024-04-26T00:24:37.739226+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:28:10.321141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.738848+0000","last_change":"2024-04-26T00:24:24.485307+0000","last_active":"2024-04-26T00:24:37.738848+0000","last_peered":"2024-04-26T00:24:37.738848+0000","last_clean":"2024-04-26T00:24:37.738848+0000","last_became_active":"2024-04-26T00:24:24.484577+0000","last_became_peered":"2024-04-26T00:24:24.484577+0000","last_unstale":"2024-04-26T00:24:37.738848+0000","last_undegraded":"2024-04-26T00:24:37.738848+0000","last_fullsized":"2024-04-26T00:24:37.738848+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:22:47.688275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.744120+0000","last_change":"2024-04-26T00:24:37.744120+0000","last_active":"2024-04-26T00:24:37.744120+0000","last_peered":"2024-04-26T00:24:37.744120+0000","last_clean":"2024-04-26T00:24:37.744120+0000","last_became_active":"2024-04-26T00:24:37.743699+0000","last_became_peered":"2024-04-26T00:24:37.743699+0000","last_unstale":"2024-04-26T00:24:37.744120+0000","last_undegraded":"2024-04-26T00:24:37.744120+0000","last_fullsized":"2024-04-26T00:24:37.744120+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:28.802989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"17'1","reported_seq":19,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-04-26T00:24:37.739310+0000","last_change":"2024-04-26T00:24:33.524248+0000","last_active":"2024-04-26T00:24:37.739310+0000","last_peered":"2024-04-26T00:24:37.739310+0000","last_clean":"2024-04-26T00:24:37.739310+0000","last_became_active":"2024-04-26T00:24:33.523571+0000","last_became_peered":"2024-04-26T00:24:33.523571+0000","last_unstale":"2024-04-26T00:24:37.739310+0000","last_undegraded":"2024-04-26T00:24:37.739310+0000","last_fullsized":"2024-04-26T00:24:37.739310+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:40:31.723150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738376,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27180,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344660,"statfs":{"total":96636764160,"available":96608931840,"internally_reserved":0,"allocated":159744,"data_stored":42217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738376,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":156,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":159744,"data_stored":42217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:43.932 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-04-26T00:24:43.932 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-04-26T00:24:43.932 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-04-26T00:24:43.979 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-04-26T00:24:43.980 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-04-26T00:24:44.025 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_1 2024-04-26T00:24:44.025 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_1 2024-04-26T00:24:44.070 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-04-26T00:24:44.070 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-04-26T00:24:44.156 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-04-26T00:24:44.156 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-04-26T00:24:44.203 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_2 2024-04-26T00:24:44.203 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_2 2024-04-26T00:24:44.248 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-04-26T00:24:44.248 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-04-26T00:24:44.294 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-04-26T00:24:44.295 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-04-26T00:24:44.341 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_3 2024-04-26T00:24:44.341 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_3 2024-04-26T00:24:44.387 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-04-26T00:24:44.387 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-04-26T00:24:44.433 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-04-26T00:24:44.433 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-04-26T00:24:44.479 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_4 2024-04-26T00:24:44.479 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_4 2024-04-26T00:24:44.524 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-04-26T00:24:44.524 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-04-26T00:24:44.570 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-04-26T00:24:44.570 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-04-26T00:24:44.615 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_5 2024-04-26T00:24:44.616 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_5 2024-04-26T00:24:44.660 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-04-26T00:24:44.661 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-04-26T00:24:44.706 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-04-26T00:24:44.707 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-04-26T00:24:44.758 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_6 2024-04-26T00:24:44.758 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_6 2024-04-26T00:24:44.804 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-04-26T00:24:44.804 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-04-26T00:24:44.850 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-04-26T00:24:44.850 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-04-26T00:24:44.896 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_7 2024-04-26T00:24:44.896 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_7 2024-04-26T00:24:44.941 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-04-26T00:24:44.941 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-04-26T00:24:44.987 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-04-26T00:24:44.987 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-04-26T00:24:45.034 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_8 2024-04-26T00:24:45.034 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_8 2024-04-26T00:24:45.079 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-04-26T00:24:45.079 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-04-26T00:24:45.125 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-04-26T00:24:45.125 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-04-26T00:24:45.171 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_9 2024-04-26T00:24:45.171 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_9 2024-04-26T00:24:45.216 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-04-26T00:24:45.334 INFO:teuthology.orchestra.run.smithi072.stdout:{} 2024-04-26T00:24:45.334 INFO:teuthology.orchestra.run.smithi072.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-04-26T00:24:45.344 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T00:24:45.345 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:24:45.381 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T00:24:45.381 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-04-26T00:24:45.712 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:24:45.734+0000 7ffb0b211640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-04-26T00:24:45.742 INFO:teuthology.orchestra.run.smithi072.stderr:marked down osd.0. 2024-04-26T00:24:45.755 INFO:tasks.util.rados:rados -p unfounddel_pool put new_1 /etc/fstab 2024-04-26T00:24:45.755 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_1 /etc/fstab 2024-04-26T00:24:46.752 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-04-26T00:24:46.752 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-04-26T00:24:46.802 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-04-26T00:24:46.802 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-04-26T00:24:46.850 INFO:tasks.util.rados:rados -p unfounddel_pool put new_2 /etc/fstab 2024-04-26T00:24:46.850 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_2 /etc/fstab 2024-04-26T00:24:46.896 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-04-26T00:24:46.896 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-04-26T00:24:46.941 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-04-26T00:24:46.941 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-04-26T00:24:46.987 INFO:tasks.util.rados:rados -p unfounddel_pool put new_3 /etc/fstab 2024-04-26T00:24:46.987 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_3 /etc/fstab 2024-04-26T00:24:47.032 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-04-26T00:24:47.033 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-04-26T00:24:47.079 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-04-26T00:24:47.079 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-04-26T00:24:47.124 INFO:tasks.util.rados:rados -p unfounddel_pool put new_4 /etc/fstab 2024-04-26T00:24:47.124 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_4 /etc/fstab 2024-04-26T00:24:47.170 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-04-26T00:24:47.171 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-04-26T00:24:47.216 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-04-26T00:24:47.216 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-04-26T00:24:47.263 INFO:tasks.util.rados:rados -p unfounddel_pool put new_5 /etc/fstab 2024-04-26T00:24:47.263 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_5 /etc/fstab 2024-04-26T00:24:47.309 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-04-26T00:24:47.309 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-04-26T00:24:47.355 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-04-26T00:24:47.355 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-04-26T00:24:47.400 INFO:tasks.util.rados:rados -p unfounddel_pool put new_6 /etc/fstab 2024-04-26T00:24:47.401 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_6 /etc/fstab 2024-04-26T00:24:47.446 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-04-26T00:24:47.446 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-04-26T00:24:47.492 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-04-26T00:24:47.492 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-04-26T00:24:47.538 INFO:tasks.util.rados:rados -p unfounddel_pool put new_7 /etc/fstab 2024-04-26T00:24:47.538 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_7 /etc/fstab 2024-04-26T00:24:47.584 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-04-26T00:24:47.584 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-04-26T00:24:47.630 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-04-26T00:24:47.630 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-04-26T00:24:47.676 INFO:tasks.util.rados:rados -p unfounddel_pool put new_8 /etc/fstab 2024-04-26T00:24:47.676 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_8 /etc/fstab 2024-04-26T00:24:47.723 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-04-26T00:24:47.723 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-04-26T00:24:47.769 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-04-26T00:24:47.770 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-04-26T00:24:47.818 INFO:tasks.util.rados:rados -p unfounddel_pool put new_9 /etc/fstab 2024-04-26T00:24:47.818 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_9 /etc/fstab 2024-04-26T00:24:47.864 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-04-26T00:24:47.864 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-04-26T00:24:47.910 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-04-26T00:24:47.910 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-04-26T00:24:47.956 INFO:tasks.rep_lost_unfound_delete:osd.0 command_args is foo 2024-04-26T00:24:47.956 INFO:tasks.rep_lost_unfound_delete:() 2024-04-26T00:24:47.956 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-26T00:24:47.956 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-04-26T00:24:47.958 INFO:tasks.ceph.osd.0:Started 2024-04-26T00:24:47.958 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:no valid command found; 10 closest matches: 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:0 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:1 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:2 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:abort 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:assert 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:config diff 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:config diff get 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:config get 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:config help [] 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:config set ... 2024-04-26T00:24:48.054 INFO:teuthology.orchestra.run.smithi072.stderr:admin_socket: invalid command 2024-04-26T00:24:48.056 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-26T00:24:48.056 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-04-26T00:24:48.545 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:24:48.566+0000 7fa0f4622640 -1 Falling back to public interface 2024-04-26T00:24:51.979 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:24:52.001+0000 7fa0f4622640 -1 osd.0 18 log_to_monitors true 2024-04-26T00:24:52.815 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:24:52.837+0000 7fa0ebd1e640 -1 osd.0 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T00:24:53.057 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-26T00:24:53.171 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:24:53.171 INFO:teuthology.orchestra.run.smithi072.stdout: "ops": [], 2024-04-26T00:24:53.171 INFO:teuthology.orchestra.run.smithi072.stdout: "num_ops": 0 2024-04-26T00:24:53.171 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:24:53.180 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-04-26T00:24:53.470 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:24:53.491+0000 7ffb0b211640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-04-26T00:24:53.470 INFO:teuthology.orchestra.run.smithi072.stderr:osd.0 is already in. 2024-04-26T00:24:53.482 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.0 to be up 2024-04-26T00:24:53.483 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:24:53.763 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:53.764 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":21,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:24:52.827004+0000","last_up_change":"2024-04-26T00:24:45.758849+0000","last_in_change":"2024-04-26T00:24:36.533919+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3.0299999713897705,"score_stable":3.0299999713897705,"optimal_score":0.33000001311302185,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-04-26T00:24:31.936865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3.0299999713897705,"score_stable":3.0299999713897705,"optimal_score":0.33000001311302185,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":21990784}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":21990784}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":21990784}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":21990784}]},"public_addr":"172.21.15.72:6803/21990784","cluster_addr":"172.21.15.72:6805/21990784","heartbeat_back_addr":"172.21.15.72:6807/21990784","heartbeat_front_addr":"172.21.15.72:6806/21990784","state":["exists"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":1686523567}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":1686523567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":1686523567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":1686523567}]},"public_addr":"172.21.15.72:6808/1686523567","cluster_addr":"172.21.15.72:6809/1686523567","heartbeat_back_addr":"172.21.15.72:6811/1686523567","heartbeat_front_addr":"172.21.15.72:6810/1686523567","state":["exists","up"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":16,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":872899250}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":872899250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":872899250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":872899250}]},"public_addr":"172.21.15.72:6800/872899250","cluster_addr":"172.21.15.72:6801/872899250","heartbeat_back_addr":"172.21.15.72:6804/872899250","heartbeat_front_addr":"172.21.15.72:6802/872899250","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":19},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":65536,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":16}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:24:56.777 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:24:57.070 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:57.071 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":23,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:24:54.832562+0000","last_up_change":"2024-04-26T00:24:53.831507+0000","last_in_change":"2024-04-26T00:24:36.533919+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-04-26T00:24:31.936865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":22,"up_thru":22,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":2078650556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":2078650556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":2078650556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":2078650556}]},"public_addr":"172.21.15.72:6800/2078650556","cluster_addr":"172.21.15.72:6801/2078650556","heartbeat_back_addr":"172.21.15.72:6803/2078650556","heartbeat_front_addr":"172.21.15.72:6802/2078650556","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":1686523567}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":1686523567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":1686523567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":1686523567}]},"public_addr":"172.21.15.72:6808/1686523567","cluster_addr":"172.21.15.72:6809/1686523567","heartbeat_back_addr":"172.21.15.72:6811/1686523567","heartbeat_front_addr":"172.21.15.72:6810/1686523567","state":["exists","up"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":13,"down_at":16,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":872899250}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":872899250}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":872899250}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":872899250}]},"public_addr":"172.21.15.72:6800/872899250","cluster_addr":"172.21.15.72:6801/872899250","heartbeat_back_addr":"172.21.15.72:6804/872899250","heartbeat_front_addr":"172.21.15.72:6802/872899250","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":19},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":65536,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":16}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:24:57.083 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.0 is up 2024-04-26T00:24:57.084 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T00:24:57.084 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-26T00:24:57.216 INFO:teuthology.orchestra.run.smithi072.stdout:94489280514 2024-04-26T00:24:57.217 INFO:teuthology.orchestra.run.smithi072.stdout:34359738380 2024-04-26T00:24:57.218 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:57.219 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:57.509 INFO:teuthology.orchestra.run.smithi072.stdout:1 2024-04-26T00:24:57.513 INFO:teuthology.orchestra.run.smithi072.stdout:34359738379 2024-04-26T00:24:57.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 94489280514 got 1 for osd.0 2024-04-26T00:24:57.527 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738380 got 34359738379 for osd.1 2024-04-26T00:24:58.523 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:24:58.528 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-26T00:24:58.865 INFO:teuthology.orchestra.run.smithi072.stdout:94489280515 2024-04-26T00:24:58.878 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 94489280514 got 94489280515 for osd.0 2024-04-26T00:24:58.878 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:58.896 INFO:teuthology.orchestra.run.smithi072.stdout:34359738380 2024-04-26T00:24:58.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738380 got 34359738380 for osd.1 2024-04-26T00:24:58.908 DEBUG:teuthology.parallel:result is None 2024-04-26T00:24:58.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-04-26T00:24:58.908 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:24:59.224 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:59.225 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:24:59.236 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2024-04-26T00:24:58.403272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55032,"kb_used_data":536,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688648,"statfs":{"total":193273528320,"available":193217175552,"internally_reserved":0,"allocated":548864,"data_stored":132790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5998,"internal_metadata":55765138},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001601"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.849304+0000","last_change":"2024-04-26T00:24:54.849304+0000","last_active":"2024-04-26T00:24:54.849304+0000","last_peered":"2024-04-26T00:24:54.849304+0000","last_clean":"2024-04-26T00:24:54.849304+0000","last_became_active":"2024-04-26T00:24:54.848908+0000","last_became_peered":"2024-04-26T00:24:54.848908+0000","last_unstale":"2024-04-26T00:24:54.849304+0000","last_undegraded":"2024-04-26T00:24:54.849304+0000","last_fullsized":"2024-04-26T00:24:54.849304+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:29:31.075740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'4","reported_seq":23,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.850213+0000","last_change":"2024-04-26T00:24:54.850378+0000","last_active":"2024-04-26T00:24:54.850213+0000","last_peered":"2024-04-26T00:24:54.850213+0000","last_clean":"2024-04-26T00:24:44.453720+0000","last_became_active":"2024-04-26T00:24:54.850111+0000","last_became_peered":"2024-04-26T00:24:54.850111+0000","last_unstale":"2024-04-26T00:24:54.850213+0000","last_undegraded":"2024-04-26T00:24:54.839737+0000","last_fullsized":"2024-04-26T00:24:54.850213+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":41,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.850348+0000","last_change":"2024-04-26T00:24:54.850655+0000","last_active":"2024-04-26T00:24:54.850348+0000","last_peered":"2024-04-26T00:24:54.850348+0000","last_clean":"2024-04-26T00:24:44.961847+0000","last_became_active":"2024-04-26T00:24:54.850253+0000","last_became_peered":"2024-04-26T00:24:54.850253+0000","last_unstale":"2024-04-26T00:24:54.850348+0000","last_undegraded":"2024-04-26T00:24:54.837118+0000","last_fullsized":"2024-04-26T00:24:54.850348+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.849230+0000","last_change":"2024-04-26T00:24:54.849230+0000","last_active":"2024-04-26T00:24:54.849230+0000","last_peered":"2024-04-26T00:24:54.849230+0000","last_clean":"2024-04-26T00:24:54.849230+0000","last_became_active":"2024-04-26T00:24:54.848598+0000","last_became_peered":"2024-04-26T00:24:54.848598+0000","last_unstale":"2024-04-26T00:24:54.849230+0000","last_undegraded":"2024-04-26T00:24:54.849230+0000","last_fullsized":"2024-04-26T00:24:54.849230+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:57:36.341888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"20'5","reported_seq":34,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.849311+0000","last_change":"2024-04-26T00:24:54.849557+0000","last_active":"2024-04-26T00:24:54.849311+0000","last_peered":"2024-04-26T00:24:54.849311+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:24:54.849224+0000","last_became_peered":"2024-04-26T00:24:54.849224+0000","last_unstale":"2024-04-26T00:24:54.849311+0000","last_undegraded":"2024-04-26T00:24:54.839303+0000","last_fullsized":"2024-04-26T00:24:54.849311+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":37,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.850427+0000","last_change":"2024-04-26T00:24:54.850702+0000","last_active":"2024-04-26T00:24:54.850427+0000","last_peered":"2024-04-26T00:24:54.850427+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:24:54.850336+0000","last_became_peered":"2024-04-26T00:24:54.850336+0000","last_unstale":"2024-04-26T00:24:54.850427+0000","last_undegraded":"2024-04-26T00:24:54.837547+0000","last_fullsized":"2024-04-26T00:24:54.850427+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"20'4","reported_seq":38,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.850815+0000","last_change":"2024-04-26T00:24:54.850990+0000","last_active":"2024-04-26T00:24:54.850815+0000","last_peered":"2024-04-26T00:24:54.850815+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:24:54.850719+0000","last_became_peered":"2024-04-26T00:24:54.850719+0000","last_unstale":"2024-04-26T00:24:54.850815+0000","last_undegraded":"2024-04-26T00:24:54.838201+0000","last_fullsized":"2024-04-26T00:24:54.850815+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"20'1","reported_seq":32,"reported_epoch":23,"state":"active+recovering+degraded","last_fresh":"2024-04-26T00:24:54.847209+0000","last_change":"2024-04-26T00:24:54.853872+0000","last_active":"2024-04-26T00:24:54.847209+0000","last_peered":"2024-04-26T00:24:54.847209+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:24:54.846818+0000","last_became_peered":"2024-04-26T00:24:54.846818+0000","last_unstale":"2024-04-26T00:24:54.847209+0000","last_undegraded":"2024-04-26T00:24:54.838953+0000","last_fullsized":"2024-04-26T00:24:54.847209+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.850403+0000","last_change":"2024-04-26T00:24:54.850403+0000","last_active":"2024-04-26T00:24:54.850403+0000","last_peered":"2024-04-26T00:24:54.850403+0000","last_clean":"2024-04-26T00:24:54.850403+0000","last_became_active":"2024-04-26T00:24:54.850102+0000","last_became_peered":"2024-04-26T00:24:54.850102+0000","last_unstale":"2024-04-26T00:24:54.850403+0000","last_undegraded":"2024-04-26T00:24:54.850403+0000","last_fullsized":"2024-04-26T00:24:54.850403+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:40.707218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"20'5","reported_seq":43,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.843359+0000","last_change":"2024-04-26T00:24:54.843642+0000","last_active":"2024-04-26T00:24:54.843359+0000","last_peered":"2024-04-26T00:24:54.843359+0000","last_clean":"2024-04-26T00:24:45.008203+0000","last_became_active":"2024-04-26T00:24:54.843283+0000","last_became_peered":"2024-04-26T00:24:54.843283+0000","last_unstale":"2024-04-26T00:24:54.843359+0000","last_undegraded":"2024-04-26T00:24:54.837903+0000","last_fullsized":"2024-04-26T00:24:54.843359+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.842678+0000","last_change":"2024-04-26T00:24:54.842678+0000","last_active":"2024-04-26T00:24:54.842678+0000","last_peered":"2024-04-26T00:24:54.842678+0000","last_clean":"2024-04-26T00:24:54.842678+0000","last_became_active":"2024-04-26T00:24:54.842422+0000","last_became_peered":"2024-04-26T00:24:54.842422+0000","last_unstale":"2024-04-26T00:24:54.842678+0000","last_undegraded":"2024-04-26T00:24:54.842678+0000","last_fullsized":"2024-04-26T00:24:54.842678+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:39:30.381107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"20'6","reported_seq":31,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.848545+0000","last_change":"2024-04-26T00:24:54.848782+0000","last_active":"2024-04-26T00:24:54.848545+0000","last_peered":"2024-04-26T00:24:54.848545+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:24:54.848455+0000","last_became_peered":"2024-04-26T00:24:54.848455+0000","last_unstale":"2024-04-26T00:24:54.848545+0000","last_undegraded":"2024-04-26T00:24:54.840627+0000","last_fullsized":"2024-04-26T00:24:54.848545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":20,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.848877+0000","last_change":"2024-04-26T00:24:54.848877+0000","last_active":"2024-04-26T00:24:54.848877+0000","last_peered":"2024-04-26T00:24:54.848877+0000","last_clean":"2024-04-26T00:24:54.848877+0000","last_became_active":"2024-04-26T00:24:54.848439+0000","last_became_peered":"2024-04-26T00:24:54.848439+0000","last_unstale":"2024-04-26T00:24:54.848877+0000","last_undegraded":"2024-04-26T00:24:54.848877+0000","last_fullsized":"2024-04-26T00:24:54.848877+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:15:37.325178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'3","reported_seq":31,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.850915+0000","last_change":"2024-04-26T00:24:54.851103+0000","last_active":"2024-04-26T00:24:54.850915+0000","last_peered":"2024-04-26T00:24:54.850915+0000","last_clean":"2024-04-26T00:24:44.315138+0000","last_became_active":"2024-04-26T00:24:54.850813+0000","last_became_peered":"2024-04-26T00:24:54.850813+0000","last_unstale":"2024-04-26T00:24:54.850915+0000","last_undegraded":"2024-04-26T00:24:54.838273+0000","last_fullsized":"2024-04-26T00:24:54.850915+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":45,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.851920+0000","last_change":"2024-04-26T00:24:54.851920+0000","last_active":"2024-04-26T00:24:54.851920+0000","last_peered":"2024-04-26T00:24:54.851920+0000","last_clean":"2024-04-26T00:24:54.851920+0000","last_became_active":"2024-04-26T00:24:54.851575+0000","last_became_peered":"2024-04-26T00:24:54.851575+0000","last_unstale":"2024-04-26T00:24:54.851920+0000","last_undegraded":"2024-04-26T00:24:54.851920+0000","last_fullsized":"2024-04-26T00:24:54.851920+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:59:04.006840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":33,"reported_epoch":23,"state":"active+recovering+degraded","last_fresh":"2024-04-26T00:24:54.843490+0000","last_change":"2024-04-26T00:24:54.851644+0000","last_active":"2024-04-26T00:24:54.843490+0000","last_peered":"2024-04-26T00:24:54.843490+0000","last_clean":"2024-04-26T00:24:44.681464+0000","last_became_active":"2024-04-26T00:24:54.843226+0000","last_became_peered":"2024-04-26T00:24:54.843226+0000","last_unstale":"2024-04-26T00:24:54.843490+0000","last_undegraded":"2024-04-26T00:24:54.836155+0000","last_fullsized":"2024-04-26T00:24:54.843490+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"20'1","reported_seq":34,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.851511+0000","last_change":"2024-04-26T00:24:54.852133+0000","last_active":"2024-04-26T00:24:54.851511+0000","last_peered":"2024-04-26T00:24:54.851511+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:24:54.851413+0000","last_became_peered":"2024-04-26T00:24:54.851413+0000","last_unstale":"2024-04-26T00:24:54.851511+0000","last_undegraded":"2024-04-26T00:24:54.837589+0000","last_fullsized":"2024-04-26T00:24:54.851511+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.851047+0000","last_change":"2024-04-26T00:24:54.851047+0000","last_active":"2024-04-26T00:24:54.851047+0000","last_peered":"2024-04-26T00:24:54.851047+0000","last_clean":"2024-04-26T00:24:54.851047+0000","last_became_active":"2024-04-26T00:24:54.850727+0000","last_became_peered":"2024-04-26T00:24:54.850727+0000","last_unstale":"2024-04-26T00:24:54.851047+0000","last_undegraded":"2024-04-26T00:24:54.851047+0000","last_fullsized":"2024-04-26T00:24:54.851047+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:17:18.932960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'5","reported_seq":39,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.850541+0000","last_change":"2024-04-26T00:24:54.850815+0000","last_active":"2024-04-26T00:24:54.850541+0000","last_peered":"2024-04-26T00:24:54.850541+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:24:54.850306+0000","last_became_peered":"2024-04-26T00:24:54.850306+0000","last_unstale":"2024-04-26T00:24:54.850541+0000","last_undegraded":"2024-04-26T00:24:54.837697+0000","last_fullsized":"2024-04-26T00:24:54.850541+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.849196+0000","last_change":"2024-04-26T00:24:54.849196+0000","last_active":"2024-04-26T00:24:54.849196+0000","last_peered":"2024-04-26T00:24:54.849196+0000","last_clean":"2024-04-26T00:24:54.849196+0000","last_became_active":"2024-04-26T00:24:54.848245+0000","last_became_peered":"2024-04-26T00:24:54.848245+0000","last_unstale":"2024-04-26T00:24:54.849196+0000","last_undegraded":"2024-04-26T00:24:54.849196+0000","last_fullsized":"2024-04-26T00:24:54.849196+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:05:09.388414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"20'2","reported_seq":35,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.845866+0000","last_change":"2024-04-26T00:24:54.846102+0000","last_active":"2024-04-26T00:24:54.845866+0000","last_peered":"2024-04-26T00:24:54.845866+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:24:54.845772+0000","last_became_peered":"2024-04-26T00:24:54.845772+0000","last_unstale":"2024-04-26T00:24:54.845866+0000","last_undegraded":"2024-04-26T00:24:54.837611+0000","last_fullsized":"2024-04-26T00:24:54.845866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.850801+0000","last_change":"2024-04-26T00:24:54.850801+0000","last_active":"2024-04-26T00:24:54.850801+0000","last_peered":"2024-04-26T00:24:54.850801+0000","last_clean":"2024-04-26T00:24:54.850801+0000","last_became_active":"2024-04-26T00:24:54.850350+0000","last_became_peered":"2024-04-26T00:24:54.850350+0000","last_unstale":"2024-04-26T00:24:54.850801+0000","last_undegraded":"2024-04-26T00:24:54.850801+0000","last_fullsized":"2024-04-26T00:24:54.850801+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:56:16.554737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-04-26T00:24:54.852375+0000","last_change":"2024-04-26T00:24:54.852375+0000","last_active":"2024-04-26T00:24:54.852375+0000","last_peered":"2024-04-26T00:24:54.852375+0000","last_clean":"2024-04-26T00:24:54.852375+0000","last_became_active":"2024-04-26T00:24:54.851828+0000","last_became_peered":"2024-04-26T00:24:54.851828+0000","last_unstale":"2024-04-26T00:24:54.852375+0000","last_undegraded":"2024-04-26T00:24:54.852375+0000","last_fullsized":"2024-04-26T00:24:54.852375+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:46:46.269449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'9","reported_seq":43,"reported_epoch":23,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:24:54.851015+0000","last_change":"2024-04-26T00:24:54.851190+0000","last_active":"2024-04-26T00:24:54.851015+0000","last_peered":"2024-04-26T00:24:54.851015+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:24:54.850913+0000","last_became_peered":"2024-04-26T00:24:54.850913+0000","last_unstale":"2024-04-26T00:24:54.851015+0000","last_undegraded":"2024-04-26T00:24:54.836865+0000","last_fullsized":"2024-04-26T00:24:54.851015+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":8,"seq":34359738380,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27324,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344516,"statfs":{"total":96636764160,"available":96608784384,"internally_reserved":0,"allocated":311296,"data_stored":70742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280515,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27708,"kb_used_data":232,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344132,"statfs":{"total":96636764160,"available":96608391168,"internally_reserved":0,"allocated":237568,"data_stored":62048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28110541},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:24:59.237 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:24:59.578 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:24:59.578 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/56 objects degraded (48.214%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":45,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":23,"num_osds":3,"num_up_osds":2,"osd_up_since":1714091093,"num_in_osds":2,"osd_in_since":1714091076,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":28,"data_bytes":13524,"bytes_used":56352768,"bytes_avail":193217175552,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":56,"degraded_ratio":0.48214285714285715},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":2,"modified":"2024-04-26T00:24:38.398996+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:24:59.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'fecf292c-395a-497e-b8b4-a892b12f3dd9', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/56 objects degraded (48.214%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 45, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 23, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1714091093, 'num_in_osds': 2, 'osd_in_since': 1714091076, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 28, 'data_bytes': 13524, 'bytes_used': 56352768, 'bytes_avail': 193217175552, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 56, 'degraded_ratio': 0.48214285714285715}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 2, 'modified': '2024-04-26T00:24:38.398996+0000', 'services': {'mgr': {'daemons': {'summary': '', 'x': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'mon': {'daemons': {'summary': '', 'a': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'b': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'c': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '1': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-04-26T00:24:59.591 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-04-26T00:24:59.591 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T00:24:59.591 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:24:59.623 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T00:24:59.624 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-04-26T00:24:59.960 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:24:59.981+0000 7ffb0b211640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-04-26T00:24:59.960 INFO:teuthology.orchestra.run.smithi072.stderr:osd.1 is already down. 2024-04-26T00:24:59.973 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-04-26T00:25:00.258 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:25:00.279+0000 7ffb0b211640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-04-26T00:25:00.848 INFO:teuthology.orchestra.run.smithi072.stderr:marked out osd.1. 2024-04-26T00:25:00.864 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-04-26T00:25:01.847 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.867+0000 7fa0d63c9640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-04-26T00:25:01.847 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.867+0000 7fa0d5bc8640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-04-26T00:25:01.847 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.867+0000 7fa0d93cf640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-04-26T00:25:01.847 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.867+0000 7fa0d63c9640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-04-26T00:25:01.847 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.867+0000 7fa0d4bc6640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-04-26T00:25:01.847 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.867+0000 7fa0d93cf640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-04-26T00:25:01.847 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.867+0000 7fa0d5bc8640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-04-26T00:25:01.848 INFO:teuthology.orchestra.run.smithi072.stderr:marked osd lost in epoch 24 2024-04-26T00:25:01.848 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.868+0000 7fa0d4bc6640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-04-26T00:25:01.848 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.868+0000 7fa0d43c5640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-04-26T00:25:01.849 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.868+0000 7fa0d3bc4640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-04-26T00:25:01.849 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.868+0000 7fa0d7bcc640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-04-26T00:25:01.849 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.868+0000 7fa0d73cb640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-04-26T00:25:01.849 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.868+0000 7fa0d6bca640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-04-26T00:25:01.849 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:01.868+0000 7fa0d33c3640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-04-26T00:25:01.862 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-26T00:25:01.862 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-26T00:25:01.864 INFO:tasks.ceph.osd.2:Started 2024-04-26T00:25:01.864 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T00:25:01.964 INFO:teuthology.orchestra.run.smithi072.stderr:no valid command found; 10 closest matches: 2024-04-26T00:25:01.964 INFO:teuthology.orchestra.run.smithi072.stderr:0 2024-04-26T00:25:01.964 INFO:teuthology.orchestra.run.smithi072.stderr:1 2024-04-26T00:25:01.964 INFO:teuthology.orchestra.run.smithi072.stderr:2 2024-04-26T00:25:01.964 INFO:teuthology.orchestra.run.smithi072.stderr:abort 2024-04-26T00:25:01.965 INFO:teuthology.orchestra.run.smithi072.stderr:assert 2024-04-26T00:25:01.965 INFO:teuthology.orchestra.run.smithi072.stderr:config diff 2024-04-26T00:25:01.965 INFO:teuthology.orchestra.run.smithi072.stderr:config diff get 2024-04-26T00:25:01.965 INFO:teuthology.orchestra.run.smithi072.stderr:config get 2024-04-26T00:25:01.965 INFO:teuthology.orchestra.run.smithi072.stderr:config help [] 2024-04-26T00:25:01.965 INFO:teuthology.orchestra.run.smithi072.stderr:config set ... 2024-04-26T00:25:01.965 INFO:teuthology.orchestra.run.smithi072.stderr:admin_socket: invalid command 2024-04-26T00:25:01.967 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-26T00:25:01.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-04-26T00:25:02.450 INFO:tasks.ceph.osd.2.smithi072.stderr:2024-04-26T00:25:02.471+0000 7f5265e41640 -1 Falling back to public interface 2024-04-26T00:25:05.913 INFO:tasks.ceph.osd.2.smithi072.stderr:2024-04-26T00:25:05.934+0000 7f5265e41640 -1 osd.2 14 log_to_monitors true 2024-04-26T00:25:06.920 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.940+0000 7fa0d63c9640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-04-26T00:25:06.920 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d53c7640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-04-26T00:25:06.920 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0da3d1640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d73cb640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d3bc4640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d7bcc640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d9bd0640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d8bce640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d4bc6640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d5bc8640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d93cf640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d43c5640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d33c3640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-04-26T00:25:06.922 INFO:tasks.ceph.osd.0.smithi072.stderr:2024-04-26T00:25:06.941+0000 7fa0d2bc2640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-04-26T00:25:06.925 INFO:tasks.ceph.osd.2.smithi072.stderr:2024-04-26T00:25:06.946+0000 7f525dbff640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T00:25:06.968 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-26T00:25:07.082 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:07.082 INFO:teuthology.orchestra.run.smithi072.stdout: "ops": [], 2024-04-26T00:25:07.082 INFO:teuthology.orchestra.run.smithi072.stdout: "num_ops": 0 2024-04-26T00:25:07.082 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:07.091 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-04-26T00:25:07.378 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:25:07.399+0000 7ffb0b211640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-04-26T00:25:07.917 INFO:teuthology.orchestra.run.smithi072.stderr:marked in osd.2. 2024-04-26T00:25:07.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.2 to be up 2024-04-26T00:25:07.936 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:25:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:08.221 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":28,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:25:07.935121+0000","last_up_change":"2024-04-26T00:25:07.935121+0000","last_in_change":"2024-04-26T00:25:07.935121+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-04-26T00:24:31.936865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":22,"up_thru":24,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":2078650556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":2078650556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":2078650556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":2078650556}]},"public_addr":"172.21.15.72:6800/2078650556","cluster_addr":"172.21.15.72:6801/2078650556","heartbeat_back_addr":"172.21.15.72:6803/2078650556","heartbeat_front_addr":"172.21.15.72:6802/2078650556","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":22,"down_at":24,"lost_at":24,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":1686523567}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":1686523567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":1686523567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":1686523567}]},"public_addr":"172.21.15.72:6808/1686523567","cluster_addr":"172.21.15.72:6809/1686523567","heartbeat_back_addr":"172.21.15.72:6811/1686523567","heartbeat_front_addr":"172.21.15.72:6810/1686523567","state":["exists"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":14,"up_from":28,"up_thru":13,"down_at":16,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":3995376740}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":3995376740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":3995376740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":3995376740}]},"public_addr":"172.21.15.72:6804/3995376740","cluster_addr":"172.21.15.72:6805/3995376740","heartbeat_back_addr":"172.21.15.72:6807/3995376740","heartbeat_front_addr":"172.21.15.72:6806/3995376740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":19},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":65536,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":16}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:25:08.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.2 is up 2024-04-26T00:25:08.234 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T00:25:08.234 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T00:25:08.364 INFO:teuthology.orchestra.run.smithi072.stdout:120259084291 2024-04-26T00:25:08.365 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:25:08.366 INFO:teuthology.orchestra.run.smithi072.stdout:94489280518 2024-04-26T00:25:08.366 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:25:08.688 INFO:teuthology.orchestra.run.smithi072.stdout:0 2024-04-26T00:25:08.689 INFO:teuthology.orchestra.run.smithi072.stdout:94489280516 2024-04-26T00:25:08.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 120259084291 got 0 for osd.2 2024-04-26T00:25:08.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 94489280518 got 94489280516 for osd.0 2024-04-26T00:25:09.702 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:25:09.703 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:25:10.007 INFO:teuthology.orchestra.run.smithi072.stdout:120259084291 2024-04-26T00:25:10.007 INFO:teuthology.orchestra.run.smithi072.stdout:94489280518 2024-04-26T00:25:10.020 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 120259084291 got 120259084291 for osd.2 2024-04-26T00:25:10.020 DEBUG:teuthology.parallel:result is None 2024-04-26T00:25:10.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 94489280518 got 94489280518 for osd.0 2024-04-26T00:25:10.021 DEBUG:teuthology.parallel:result is None 2024-04-26T00:25:10.021 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-04-26T00:25:10.021 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:10.294 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:10.294 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:10.306 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2024-04-26T00:25:08.962049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55496,"kb_used_data":496,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688184,"statfs":{"total":193273528320,"available":193216700416,"internally_reserved":0,"allocated":507904,"data_stored":146899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":56286621},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":20,"num_objects_missing":20,"num_objects_degraded":-27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.557840"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":32,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.943251+0000","last_change":"2024-04-26T00:25:07.952327+0000","last_active":"2024-04-26T00:25:07.952327+0000","last_peered":"2024-04-26T00:25:06.943251+0000","last_clean":"2024-04-26T00:24:54.853202+0000","last_became_active":"2024-04-26T00:25:00.872619+0000","last_became_peered":"2024-04-26T00:25:00.872619+0000","last_unstale":"2024-04-26T00:25:06.943251+0000","last_undegraded":"2024-04-26T00:25:06.943251+0000","last_fullsized":"2024-04-26T00:25:00.869702+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'4","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.942656+0000","last_change":"2024-04-26T00:25:07.962444+0000","last_active":"2024-04-26T00:25:07.962444+0000","last_peered":"2024-04-26T00:25:06.942656+0000","last_clean":"2024-04-26T00:24:44.453720+0000","last_became_active":"2024-04-26T00:25:00.871478+0000","last_became_peered":"2024-04-26T00:25:00.871478+0000","last_unstale":"2024-04-26T00:25:06.942656+0000","last_undegraded":"2024-04-26T00:25:00.869363+0000","last_fullsized":"2024-04-26T00:25:00.869243+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":7,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.963014+0000","last_change":"2024-04-26T00:25:07.963014+0000","last_active":"2024-04-26T00:25:07.963014+0000","last_peered":"2024-04-26T00:25:06.942724+0000","last_clean":"2024-04-26T00:24:44.961847+0000","last_became_active":"2024-04-26T00:25:00.872948+0000","last_became_peered":"2024-04-26T00:25:00.872948+0000","last_unstale":"2024-04-26T00:25:07.963014+0000","last_undegraded":"2024-04-26T00:25:07.963014+0000","last_fullsized":"2024-04-26T00:25:07.963014+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.960837+0000","last_change":"2024-04-26T00:25:07.959438+0000","last_active":"2024-04-26T00:25:06.947280+0000","last_peered":"2024-04-26T00:24:33.528848+0000","last_clean":"2024-04-26T00:24:33.528848+0000","last_became_active":"2024-04-26T00:24:33.528499+0000","last_became_peered":"2024-04-26T00:24:33.528499+0000","last_unstale":"2024-04-26T00:25:07.960837+0000","last_undegraded":"2024-04-26T00:25:07.960837+0000","last_fullsized":"2024-04-26T00:25:07.960837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"20'5","reported_seq":45,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942796+0000","last_change":"2024-04-26T00:25:07.950026+0000","last_active":"2024-04-26T00:25:07.950026+0000","last_peered":"2024-04-26T00:25:06.942796+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:00.872517+0000","last_became_peered":"2024-04-26T00:25:00.872517+0000","last_unstale":"2024-04-26T00:25:06.942796+0000","last_undegraded":"2024-04-26T00:25:00.869416+0000","last_fullsized":"2024-04-26T00:25:00.869301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":44,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942463+0000","last_change":"2024-04-26T00:25:07.950019+0000","last_active":"2024-04-26T00:25:07.950019+0000","last_peered":"2024-04-26T00:25:06.942463+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:00.871658+0000","last_became_peered":"2024-04-26T00:25:00.871658+0000","last_unstale":"2024-04-26T00:25:06.942463+0000","last_undegraded":"2024-04-26T00:25:00.868522+0000","last_fullsized":"2024-04-26T00:25:00.868405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"20'4","reported_seq":45,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942758+0000","last_change":"2024-04-26T00:25:07.949842+0000","last_active":"2024-04-26T00:25:07.949842+0000","last_peered":"2024-04-26T00:25:06.942758+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:00.872837+0000","last_became_peered":"2024-04-26T00:25:00.872837+0000","last_unstale":"2024-04-26T00:25:06.942758+0000","last_undegraded":"2024-04-26T00:25:00.869448+0000","last_fullsized":"2024-04-26T00:25:00.868927+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'1","reported_seq":43,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942527+0000","last_change":"2024-04-26T00:25:07.949349+0000","last_active":"2024-04-26T00:25:07.949349+0000","last_peered":"2024-04-26T00:25:06.942527+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:00.872295+0000","last_became_peered":"2024-04-26T00:25:00.872295+0000","last_unstale":"2024-04-26T00:25:06.942527+0000","last_undegraded":"2024-04-26T00:25:00.869054+0000","last_fullsized":"2024-04-26T00:25:00.868928+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.941968+0000","last_change":"2024-04-26T00:25:07.948717+0000","last_active":"2024-04-26T00:25:07.948717+0000","last_peered":"2024-04-26T00:25:06.941968+0000","last_clean":"2024-04-26T00:24:37.738607+0000","last_became_active":"2024-04-26T00:25:00.871282+0000","last_became_peered":"2024-04-26T00:25:00.871282+0000","last_unstale":"2024-04-26T00:25:06.941968+0000","last_undegraded":"2024-04-26T00:25:06.941968+0000","last_fullsized":"2024-04-26T00:25:00.868258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"20'5","reported_seq":7,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.962224+0000","last_change":"2024-04-26T00:25:07.962224+0000","last_active":"2024-04-26T00:25:07.962224+0000","last_peered":"2024-04-26T00:25:06.942833+0000","last_clean":"2024-04-26T00:24:45.008203+0000","last_became_active":"2024-04-26T00:25:00.872637+0000","last_became_peered":"2024-04-26T00:25:00.872637+0000","last_unstale":"2024-04-26T00:25:07.962224+0000","last_undegraded":"2024-04-26T00:25:07.962224+0000","last_fullsized":"2024-04-26T00:25:07.962224+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.958983+0000","last_change":"2024-04-26T00:25:06.945127+0000","last_active":"2024-04-26T00:25:06.945127+0000","last_peered":"2024-04-26T00:24:24.489079+0000","last_clean":"2024-04-26T00:24:24.489079+0000","last_became_active":"2024-04-26T00:24:24.488765+0000","last_became_peered":"2024-04-26T00:24:24.488765+0000","last_unstale":"2024-04-26T00:25:07.958983+0000","last_undegraded":"2024-04-26T00:25:07.958983+0000","last_fullsized":"2024-04-26T00:25:07.958983+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"20'6","reported_seq":42,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942901+0000","last_change":"2024-04-26T00:25:07.951704+0000","last_active":"2024-04-26T00:25:07.951704+0000","last_peered":"2024-04-26T00:25:06.942901+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:00.871954+0000","last_became_peered":"2024-04-26T00:25:00.871954+0000","last_unstale":"2024-04-26T00:25:06.942901+0000","last_undegraded":"2024-04-26T00:25:00.868733+0000","last_fullsized":"2024-04-26T00:25:00.868614+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942454+0000","last_change":"2024-04-26T00:25:07.950511+0000","last_active":"2024-04-26T00:25:07.950511+0000","last_peered":"2024-04-26T00:25:06.942454+0000","last_clean":"2024-04-26T00:24:54.849972+0000","last_became_active":"2024-04-26T00:25:00.871530+0000","last_became_peered":"2024-04-26T00:25:00.871530+0000","last_unstale":"2024-04-26T00:25:06.942454+0000","last_undegraded":"2024-04-26T00:25:06.942454+0000","last_fullsized":"2024-04-26T00:25:00.868319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'3","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.942837+0000","last_change":"2024-04-26T00:25:07.962548+0000","last_active":"2024-04-26T00:25:07.962548+0000","last_peered":"2024-04-26T00:25:06.942837+0000","last_clean":"2024-04-26T00:24:44.315138+0000","last_became_active":"2024-04-26T00:25:00.872062+0000","last_became_peered":"2024-04-26T00:25:00.872062+0000","last_unstale":"2024-04-26T00:25:06.942837+0000","last_undegraded":"2024-04-26T00:25:00.868585+0000","last_fullsized":"2024-04-26T00:25:00.868458+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":5,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.942326+0000","last_change":"2024-04-26T00:25:08.374807+0000","last_active":"2024-04-26T00:25:08.374807+0000","last_peered":"2024-04-26T00:25:06.942326+0000","last_clean":"2024-04-26T00:24:37.749174+0000","last_became_active":"2024-04-26T00:25:00.871471+0000","last_became_peered":"2024-04-26T00:25:00.871471+0000","last_unstale":"2024-04-26T00:25:06.942326+0000","last_undegraded":"2024-04-26T00:25:06.942326+0000","last_fullsized":"2024-04-26T00:25:00.868951+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.942058+0000","last_change":"2024-04-26T00:25:07.962620+0000","last_active":"2024-04-26T00:25:07.962620+0000","last_peered":"2024-04-26T00:25:06.942058+0000","last_clean":"2024-04-26T00:24:44.681464+0000","last_became_active":"2024-04-26T00:25:00.871425+0000","last_became_peered":"2024-04-26T00:25:00.871425+0000","last_unstale":"2024-04-26T00:25:06.942058+0000","last_undegraded":"2024-04-26T00:25:00.868513+0000","last_fullsized":"2024-04-26T00:25:00.868262+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"20'1","reported_seq":41,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.943136+0000","last_change":"2024-04-26T00:25:07.951826+0000","last_active":"2024-04-26T00:25:07.951826+0000","last_peered":"2024-04-26T00:25:06.943136+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:00.871989+0000","last_became_peered":"2024-04-26T00:25:00.871989+0000","last_unstale":"2024-04-26T00:25:06.943136+0000","last_undegraded":"2024-04-26T00:25:00.869337+0000","last_fullsized":"2024-04-26T00:25:00.869221+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942529+0000","last_change":"2024-04-26T00:25:07.951196+0000","last_active":"2024-04-26T00:25:07.951196+0000","last_peered":"2024-04-26T00:25:06.942529+0000","last_clean":"2024-04-26T00:24:37.739248+0000","last_became_active":"2024-04-26T00:25:00.873831+0000","last_became_peered":"2024-04-26T00:25:00.873831+0000","last_unstale":"2024-04-26T00:25:06.942529+0000","last_undegraded":"2024-04-26T00:25:06.942529+0000","last_fullsized":"2024-04-26T00:25:00.870276+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"20'5","reported_seq":46,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.943029+0000","last_change":"2024-04-26T00:25:07.952620+0000","last_active":"2024-04-26T00:25:07.952620+0000","last_peered":"2024-04-26T00:25:06.943029+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:00.872930+0000","last_became_peered":"2024-04-26T00:25:00.872930+0000","last_unstale":"2024-04-26T00:25:06.943029+0000","last_undegraded":"2024-04-26T00:25:00.869377+0000","last_fullsized":"2024-04-26T00:25:00.869255+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.959088+0000","last_change":"2024-04-26T00:25:06.945210+0000","last_active":"2024-04-26T00:25:06.945210+0000","last_peered":"2024-04-26T00:24:24.489032+0000","last_clean":"2024-04-26T00:24:24.489032+0000","last_became_active":"2024-04-26T00:24:24.488722+0000","last_became_peered":"2024-04-26T00:24:24.488722+0000","last_unstale":"2024-04-26T00:25:07.959088+0000","last_undegraded":"2024-04-26T00:25:07.959088+0000","last_fullsized":"2024-04-26T00:25:07.959088+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"20'2","reported_seq":42,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942717+0000","last_change":"2024-04-26T00:25:07.951420+0000","last_active":"2024-04-26T00:25:07.951420+0000","last_peered":"2024-04-26T00:25:06.942717+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:00.873334+0000","last_became_peered":"2024-04-26T00:25:00.873334+0000","last_unstale":"2024-04-26T00:25:06.942717+0000","last_undegraded":"2024-04-26T00:25:00.870411+0000","last_fullsized":"2024-04-26T00:25:00.870287+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942456+0000","last_change":"2024-04-26T00:25:07.951406+0000","last_active":"2024-04-26T00:25:07.951406+0000","last_peered":"2024-04-26T00:25:06.942456+0000","last_clean":"2024-04-26T00:24:37.738889+0000","last_became_active":"2024-04-26T00:25:00.871408+0000","last_became_peered":"2024-04-26T00:25:00.871408+0000","last_unstale":"2024-04-26T00:25:06.942456+0000","last_undegraded":"2024-04-26T00:25:06.942456+0000","last_fullsized":"2024-04-26T00:25:00.868809+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":5,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.943075+0000","last_change":"2024-04-26T00:25:08.374867+0000","last_active":"2024-04-26T00:25:08.374867+0000","last_peered":"2024-04-26T00:25:06.943075+0000","last_clean":"2024-04-26T00:24:37.748902+0000","last_became_active":"2024-04-26T00:25:00.871641+0000","last_became_peered":"2024-04-26T00:25:00.871641+0000","last_unstale":"2024-04-26T00:25:06.943075+0000","last_undegraded":"2024-04-26T00:25:06.943075+0000","last_fullsized":"2024-04-26T00:25:00.868901+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"20'9","reported_seq":50,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942919+0000","last_change":"2024-04-26T00:25:07.952056+0000","last_active":"2024-04-26T00:25:07.952056+0000","last_peered":"2024-04-26T00:25:06.942919+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:00.872770+0000","last_became_peered":"2024-04-26T00:25:00.872770+0000","last_unstale":"2024-04-26T00:25:06.942919+0000","last_undegraded":"2024-04-26T00:25:00.869378+0000","last_fullsized":"2024-04-26T00:25:00.869259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280518,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":278528,"data_stored":77590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28110541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084291,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":224,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":229376,"data_stored":69309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":28176080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:10.307 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:25:10.654 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:10.654 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":56,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":29,"num_osds":3,"num_up_osds":2,"osd_up_since":1714091107,"num_in_osds":2,"osd_in_since":1714091107,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":24}],"num_pgs":24,"num_pools":2,"num_objects":28,"data_bytes":13524,"bytes_used":56827904,"bytes_avail":193216700416,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-26T00:25:02.404228+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:25:10.667 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'fecf292c-395a-497e-b8b4-a892b12f3dd9', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 56, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 29, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1714091107, 'num_in_osds': 2, 'osd_in_since': 1714091107, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 24}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 28, 'data_bytes': 13524, 'bytes_used': 56827904, 'bytes_avail': 193216700416, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-04-26T00:25:02.404228+0000', 'services': {'mgr': {'daemons': {'summary': '', 'x': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'mon': {'daemons': {'summary': '', 'a': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'b': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'c': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-04-26T00:25:13.668 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:13.958 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:13.959 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:13.970 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2024-04-26T00:25:12.406065+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55496,"kb_used_data":496,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688184,"statfs":{"total":193273528320,"available":193216700416,"internally_reserved":0,"allocated":507904,"data_stored":146899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":56286621},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-55,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001059"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":32,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.943251+0000","last_change":"2024-04-26T00:25:07.952327+0000","last_active":"2024-04-26T00:25:07.952327+0000","last_peered":"2024-04-26T00:25:06.943251+0000","last_clean":"2024-04-26T00:24:54.853202+0000","last_became_active":"2024-04-26T00:25:00.872619+0000","last_became_peered":"2024-04-26T00:25:00.872619+0000","last_unstale":"2024-04-26T00:25:06.943251+0000","last_undegraded":"2024-04-26T00:25:06.943251+0000","last_fullsized":"2024-04-26T00:25:00.869702+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'4","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.942656+0000","last_change":"2024-04-26T00:25:07.962444+0000","last_active":"2024-04-26T00:25:07.962444+0000","last_peered":"2024-04-26T00:25:06.942656+0000","last_clean":"2024-04-26T00:24:44.453720+0000","last_became_active":"2024-04-26T00:25:00.871478+0000","last_became_peered":"2024-04-26T00:25:00.871478+0000","last_unstale":"2024-04-26T00:25:06.942656+0000","last_undegraded":"2024-04-26T00:25:00.869363+0000","last_fullsized":"2024-04-26T00:25:00.869243+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":7,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.963014+0000","last_change":"2024-04-26T00:25:07.963014+0000","last_active":"2024-04-26T00:25:07.963014+0000","last_peered":"2024-04-26T00:25:06.942724+0000","last_clean":"2024-04-26T00:24:44.961847+0000","last_became_active":"2024-04-26T00:25:00.872948+0000","last_became_peered":"2024-04-26T00:25:00.872948+0000","last_unstale":"2024-04-26T00:25:07.963014+0000","last_undegraded":"2024-04-26T00:25:07.963014+0000","last_fullsized":"2024-04-26T00:25:07.963014+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.960837+0000","last_change":"2024-04-26T00:25:07.959438+0000","last_active":"2024-04-26T00:25:06.947280+0000","last_peered":"2024-04-26T00:24:33.528848+0000","last_clean":"2024-04-26T00:24:33.528848+0000","last_became_active":"2024-04-26T00:24:33.528499+0000","last_became_peered":"2024-04-26T00:24:33.528499+0000","last_unstale":"2024-04-26T00:25:07.960837+0000","last_undegraded":"2024-04-26T00:25:07.960837+0000","last_fullsized":"2024-04-26T00:25:07.960837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"20'5","reported_seq":45,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942796+0000","last_change":"2024-04-26T00:25:07.950026+0000","last_active":"2024-04-26T00:25:07.950026+0000","last_peered":"2024-04-26T00:25:06.942796+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:00.872517+0000","last_became_peered":"2024-04-26T00:25:00.872517+0000","last_unstale":"2024-04-26T00:25:06.942796+0000","last_undegraded":"2024-04-26T00:25:00.869416+0000","last_fullsized":"2024-04-26T00:25:00.869301+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":44,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942463+0000","last_change":"2024-04-26T00:25:07.950019+0000","last_active":"2024-04-26T00:25:07.950019+0000","last_peered":"2024-04-26T00:25:06.942463+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:00.871658+0000","last_became_peered":"2024-04-26T00:25:00.871658+0000","last_unstale":"2024-04-26T00:25:06.942463+0000","last_undegraded":"2024-04-26T00:25:00.868522+0000","last_fullsized":"2024-04-26T00:25:00.868405+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"20'4","reported_seq":45,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942758+0000","last_change":"2024-04-26T00:25:07.949842+0000","last_active":"2024-04-26T00:25:07.949842+0000","last_peered":"2024-04-26T00:25:06.942758+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:00.872837+0000","last_became_peered":"2024-04-26T00:25:00.872837+0000","last_unstale":"2024-04-26T00:25:06.942758+0000","last_undegraded":"2024-04-26T00:25:00.869448+0000","last_fullsized":"2024-04-26T00:25:00.868927+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'1","reported_seq":43,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942527+0000","last_change":"2024-04-26T00:25:07.949349+0000","last_active":"2024-04-26T00:25:07.949349+0000","last_peered":"2024-04-26T00:25:06.942527+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:00.872295+0000","last_became_peered":"2024-04-26T00:25:00.872295+0000","last_unstale":"2024-04-26T00:25:06.942527+0000","last_undegraded":"2024-04-26T00:25:00.869054+0000","last_fullsized":"2024-04-26T00:25:00.868928+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.941968+0000","last_change":"2024-04-26T00:25:07.948717+0000","last_active":"2024-04-26T00:25:07.948717+0000","last_peered":"2024-04-26T00:25:06.941968+0000","last_clean":"2024-04-26T00:24:37.738607+0000","last_became_active":"2024-04-26T00:25:00.871282+0000","last_became_peered":"2024-04-26T00:25:00.871282+0000","last_unstale":"2024-04-26T00:25:06.941968+0000","last_undegraded":"2024-04-26T00:25:06.941968+0000","last_fullsized":"2024-04-26T00:25:00.868258+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"20'5","reported_seq":7,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.962224+0000","last_change":"2024-04-26T00:25:07.962224+0000","last_active":"2024-04-26T00:25:07.962224+0000","last_peered":"2024-04-26T00:25:06.942833+0000","last_clean":"2024-04-26T00:24:45.008203+0000","last_became_active":"2024-04-26T00:25:00.872637+0000","last_became_peered":"2024-04-26T00:25:00.872637+0000","last_unstale":"2024-04-26T00:25:07.962224+0000","last_undegraded":"2024-04-26T00:25:07.962224+0000","last_fullsized":"2024-04-26T00:25:07.962224+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.958983+0000","last_change":"2024-04-26T00:25:06.945127+0000","last_active":"2024-04-26T00:25:06.945127+0000","last_peered":"2024-04-26T00:24:24.489079+0000","last_clean":"2024-04-26T00:24:24.489079+0000","last_became_active":"2024-04-26T00:24:24.488765+0000","last_became_peered":"2024-04-26T00:24:24.488765+0000","last_unstale":"2024-04-26T00:25:07.958983+0000","last_undegraded":"2024-04-26T00:25:07.958983+0000","last_fullsized":"2024-04-26T00:25:07.958983+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"20'6","reported_seq":42,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942901+0000","last_change":"2024-04-26T00:25:07.951704+0000","last_active":"2024-04-26T00:25:07.951704+0000","last_peered":"2024-04-26T00:25:06.942901+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:00.871954+0000","last_became_peered":"2024-04-26T00:25:00.871954+0000","last_unstale":"2024-04-26T00:25:06.942901+0000","last_undegraded":"2024-04-26T00:25:00.868733+0000","last_fullsized":"2024-04-26T00:25:00.868614+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942454+0000","last_change":"2024-04-26T00:25:07.950511+0000","last_active":"2024-04-26T00:25:07.950511+0000","last_peered":"2024-04-26T00:25:06.942454+0000","last_clean":"2024-04-26T00:24:54.849972+0000","last_became_active":"2024-04-26T00:25:00.871530+0000","last_became_peered":"2024-04-26T00:25:00.871530+0000","last_unstale":"2024-04-26T00:25:06.942454+0000","last_undegraded":"2024-04-26T00:25:06.942454+0000","last_fullsized":"2024-04-26T00:25:00.868319+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'3","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.942837+0000","last_change":"2024-04-26T00:25:07.962548+0000","last_active":"2024-04-26T00:25:07.962548+0000","last_peered":"2024-04-26T00:25:06.942837+0000","last_clean":"2024-04-26T00:24:44.315138+0000","last_became_active":"2024-04-26T00:25:00.872062+0000","last_became_peered":"2024-04-26T00:25:00.872062+0000","last_unstale":"2024-04-26T00:25:06.942837+0000","last_undegraded":"2024-04-26T00:25:00.868585+0000","last_fullsized":"2024-04-26T00:25:00.868458+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":5,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.942326+0000","last_change":"2024-04-26T00:25:08.374807+0000","last_active":"2024-04-26T00:25:08.374807+0000","last_peered":"2024-04-26T00:25:06.942326+0000","last_clean":"2024-04-26T00:24:37.749174+0000","last_became_active":"2024-04-26T00:25:00.871471+0000","last_became_peered":"2024-04-26T00:25:00.871471+0000","last_unstale":"2024-04-26T00:25:06.942326+0000","last_undegraded":"2024-04-26T00:25:06.942326+0000","last_fullsized":"2024-04-26T00:25:00.868951+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.942058+0000","last_change":"2024-04-26T00:25:07.962620+0000","last_active":"2024-04-26T00:25:07.962620+0000","last_peered":"2024-04-26T00:25:06.942058+0000","last_clean":"2024-04-26T00:24:44.681464+0000","last_became_active":"2024-04-26T00:25:00.871425+0000","last_became_peered":"2024-04-26T00:25:00.871425+0000","last_unstale":"2024-04-26T00:25:06.942058+0000","last_undegraded":"2024-04-26T00:25:00.868513+0000","last_fullsized":"2024-04-26T00:25:00.868262+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"20'1","reported_seq":41,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.943136+0000","last_change":"2024-04-26T00:25:07.951826+0000","last_active":"2024-04-26T00:25:07.951826+0000","last_peered":"2024-04-26T00:25:06.943136+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:00.871989+0000","last_became_peered":"2024-04-26T00:25:00.871989+0000","last_unstale":"2024-04-26T00:25:06.943136+0000","last_undegraded":"2024-04-26T00:25:00.869337+0000","last_fullsized":"2024-04-26T00:25:00.869221+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942529+0000","last_change":"2024-04-26T00:25:07.951196+0000","last_active":"2024-04-26T00:25:07.951196+0000","last_peered":"2024-04-26T00:25:06.942529+0000","last_clean":"2024-04-26T00:24:37.739248+0000","last_became_active":"2024-04-26T00:25:00.873831+0000","last_became_peered":"2024-04-26T00:25:00.873831+0000","last_unstale":"2024-04-26T00:25:06.942529+0000","last_undegraded":"2024-04-26T00:25:06.942529+0000","last_fullsized":"2024-04-26T00:25:00.870276+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"20'5","reported_seq":46,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.943029+0000","last_change":"2024-04-26T00:25:07.952620+0000","last_active":"2024-04-26T00:25:07.952620+0000","last_peered":"2024-04-26T00:25:06.943029+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:00.872930+0000","last_became_peered":"2024-04-26T00:25:00.872930+0000","last_unstale":"2024-04-26T00:25:06.943029+0000","last_undegraded":"2024-04-26T00:25:00.869377+0000","last_fullsized":"2024-04-26T00:25:00.869255+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":12,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:07.959088+0000","last_change":"2024-04-26T00:25:06.945210+0000","last_active":"2024-04-26T00:25:06.945210+0000","last_peered":"2024-04-26T00:24:24.489032+0000","last_clean":"2024-04-26T00:24:24.489032+0000","last_became_active":"2024-04-26T00:24:24.488722+0000","last_became_peered":"2024-04-26T00:24:24.488722+0000","last_unstale":"2024-04-26T00:25:07.959088+0000","last_undegraded":"2024-04-26T00:25:07.959088+0000","last_fullsized":"2024-04-26T00:25:07.959088+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"20'2","reported_seq":42,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942717+0000","last_change":"2024-04-26T00:25:07.951420+0000","last_active":"2024-04-26T00:25:07.951420+0000","last_peered":"2024-04-26T00:25:06.942717+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:00.873334+0000","last_became_peered":"2024-04-26T00:25:00.873334+0000","last_unstale":"2024-04-26T00:25:06.942717+0000","last_undegraded":"2024-04-26T00:25:00.870411+0000","last_fullsized":"2024-04-26T00:25:00.870287+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942456+0000","last_change":"2024-04-26T00:25:07.951406+0000","last_active":"2024-04-26T00:25:07.951406+0000","last_peered":"2024-04-26T00:25:06.942456+0000","last_clean":"2024-04-26T00:24:37.738889+0000","last_became_active":"2024-04-26T00:25:00.871408+0000","last_became_peered":"2024-04-26T00:25:00.871408+0000","last_unstale":"2024-04-26T00:25:06.942456+0000","last_undegraded":"2024-04-26T00:25:06.942456+0000","last_fullsized":"2024-04-26T00:25:00.868809+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":5,"reported_epoch":28,"state":"peering","last_fresh":"2024-04-26T00:25:06.943075+0000","last_change":"2024-04-26T00:25:08.374867+0000","last_active":"2024-04-26T00:25:08.374867+0000","last_peered":"2024-04-26T00:25:06.943075+0000","last_clean":"2024-04-26T00:24:37.748902+0000","last_became_active":"2024-04-26T00:25:00.871641+0000","last_became_peered":"2024-04-26T00:25:00.871641+0000","last_unstale":"2024-04-26T00:25:06.943075+0000","last_undegraded":"2024-04-26T00:25:06.943075+0000","last_fullsized":"2024-04-26T00:25:00.868901+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"20'9","reported_seq":50,"reported_epoch":27,"state":"peering","last_fresh":"2024-04-26T00:25:06.942919+0000","last_change":"2024-04-26T00:25:07.952056+0000","last_active":"2024-04-26T00:25:07.952056+0000","last_peered":"2024-04-26T00:25:06.942919+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:00.872770+0000","last_became_peered":"2024-04-26T00:25:00.872770+0000","last_unstale":"2024-04-26T00:25:06.942919+0000","last_undegraded":"2024-04-26T00:25:00.869378+0000","last_fullsized":"2024-04-26T00:25:00.869259+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280518,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27740,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344100,"statfs":{"total":96636764160,"available":96608358400,"internally_reserved":0,"allocated":278528,"data_stored":77590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28110541},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084291,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27756,"kb_used_data":224,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344084,"statfs":{"total":96636764160,"available":96608342016,"internally_reserved":0,"allocated":229376,"data_stored":69309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":28176080},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:13.972 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:25:14.313 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:14.313 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":60,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":29,"num_osds":3,"num_up_osds":2,"osd_up_since":1714091107,"num_in_osds":2,"osd_in_since":1714091107,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":24}],"num_pgs":24,"num_pools":2,"num_objects":28,"data_bytes":13524,"bytes_used":56827904,"bytes_avail":193216700416,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-26T00:25:02.404228+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:25:14.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'fecf292c-395a-497e-b8b4-a892b12f3dd9', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 60, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 29, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1714091107, 'num_in_osds': 2, 'osd_in_since': 1714091107, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 24}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 28, 'data_bytes': 13524, 'bytes_used': 56827904, 'bytes_avail': 193216700416, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-04-26T00:25:02.404228+0000', 'services': {'mgr': {'daemons': {'summary': '', 'x': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'mon': {'daemons': {'summary': '', 'a': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'b': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'c': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-04-26T00:25:14.942 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:25:14.963+0000 7ffb0da16640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-04-26T00:25:17.325 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:17.616 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:17.617 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:17.628 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-04-26T00:25:16.406890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55524,"kb_used_data":520,"kb_used_omap":8,"kb_used_meta":54967,"kb_avail":188688156,"statfs":{"total":193273528320,"available":193216671744,"internally_reserved":0,"allocated":532480,"data_stored":155784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":56286621},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.459336"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":40,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.976334+0000","last_change":"2024-04-26T00:25:08.976334+0000","last_active":"2024-04-26T00:25:08.976334+0000","last_peered":"2024-04-26T00:25:08.976334+0000","last_clean":"2024-04-26T00:25:08.976334+0000","last_became_active":"2024-04-26T00:25:08.976010+0000","last_became_peered":"2024-04-26T00:25:08.976010+0000","last_unstale":"2024-04-26T00:25:08.976334+0000","last_undegraded":"2024-04-26T00:25:08.976334+0000","last_fullsized":"2024-04-26T00:25:08.976334+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:51.146504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"20'4","reported_seq":20,"reported_epoch":29,"state":"active+recovery_unfound","last_fresh":"2024-04-26T00:25:08.977782+0000","last_change":"2024-04-26T00:25:08.978015+0000","last_active":"2024-04-26T00:25:08.977782+0000","last_peered":"2024-04-26T00:25:08.977782+0000","last_clean":"2024-04-26T00:24:44.453720+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:08.977782+0000","last_undegraded":"2024-04-26T00:25:08.968083+0000","last_fullsized":"2024-04-26T00:25:08.977782+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"20'3","reported_seq":15,"reported_epoch":29,"state":"active+recovery_unfound","last_fresh":"2024-04-26T00:25:08.978256+0000","last_change":"2024-04-26T00:25:08.978413+0000","last_active":"2024-04-26T00:25:08.978256+0000","last_peered":"2024-04-26T00:25:08.978256+0000","last_clean":"2024-04-26T00:24:44.961847+0000","last_became_active":"2024-04-26T00:25:08.973037+0000","last_became_peered":"2024-04-26T00:25:08.973037+0000","last_unstale":"2024-04-26T00:25:08.978256+0000","last_undegraded":"2024-04-26T00:25:08.968903+0000","last_fullsized":"2024-04-26T00:25:08.978256+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969528+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:08.969528+0000","last_peered":"2024-04-26T00:25:08.969528+0000","last_clean":"2024-04-26T00:25:08.969528+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:08.969528+0000","last_undegraded":"2024-04-26T00:25:08.969528+0000","last_fullsized":"2024-04-26T00:25:08.969528+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"20'5","reported_seq":51,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:08.973902+0000","last_change":"2024-04-26T00:25:08.974060+0000","last_active":"2024-04-26T00:25:08.973902+0000","last_peered":"2024-04-26T00:25:08.973902+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:08.973902+0000","last_undegraded":"2024-04-26T00:25:08.963569+0000","last_fullsized":"2024-04-26T00:25:08.973902+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"20'3","reported_seq":50,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:08.972537+0000","last_change":"2024-04-26T00:25:08.972675+0000","last_active":"2024-04-26T00:25:08.972537+0000","last_peered":"2024-04-26T00:25:08.972537+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:08.972454+0000","last_became_peered":"2024-04-26T00:25:08.972454+0000","last_unstale":"2024-04-26T00:25:08.972537+0000","last_undegraded":"2024-04-26T00:25:08.963672+0000","last_fullsized":"2024-04-26T00:25:08.972537+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"20'4","reported_seq":51,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:08.970664+0000","last_change":"2024-04-26T00:25:08.970817+0000","last_active":"2024-04-26T00:25:08.970664+0000","last_peered":"2024-04-26T00:25:08.970664+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:08.970587+0000","last_became_peered":"2024-04-26T00:25:08.970587+0000","last_unstale":"2024-04-26T00:25:08.970664+0000","last_undegraded":"2024-04-26T00:25:08.964127+0000","last_fullsized":"2024-04-26T00:25:08.970664+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"20'1","reported_seq":50,"reported_epoch":29,"state":"active+recovering+degraded","last_fresh":"2024-04-26T00:25:08.969484+0000","last_change":"2024-04-26T00:25:08.970328+0000","last_active":"2024-04-26T00:25:08.969484+0000","last_peered":"2024-04-26T00:25:08.969484+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:08.969484+0000","last_undegraded":"2024-04-26T00:25:08.963546+0000","last_fullsized":"2024-04-26T00:25:08.969484+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973776+0000","last_change":"2024-04-26T00:25:08.973776+0000","last_active":"2024-04-26T00:25:08.973776+0000","last_peered":"2024-04-26T00:25:08.973776+0000","last_clean":"2024-04-26T00:25:08.973776+0000","last_became_active":"2024-04-26T00:25:08.973531+0000","last_became_peered":"2024-04-26T00:25:08.973531+0000","last_unstale":"2024-04-26T00:25:08.973776+0000","last_undegraded":"2024-04-26T00:25:08.973776+0000","last_fullsized":"2024-04-26T00:25:08.973776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:51:51.533562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"20'5","reported_seq":15,"reported_epoch":29,"state":"active+recovery_unfound","last_fresh":"2024-04-26T00:25:08.976625+0000","last_change":"2024-04-26T00:25:08.977167+0000","last_active":"2024-04-26T00:25:08.976625+0000","last_peered":"2024-04-26T00:25:08.976625+0000","last_clean":"2024-04-26T00:24:45.008203+0000","last_became_active":"2024-04-26T00:25:08.972082+0000","last_became_peered":"2024-04-26T00:25:08.972082+0000","last_unstale":"2024-04-26T00:25:08.976625+0000","last_undegraded":"2024-04-26T00:25:08.966392+0000","last_fullsized":"2024-04-26T00:25:08.976625+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969847+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:08.969847+0000","last_peered":"2024-04-26T00:25:08.969847+0000","last_clean":"2024-04-26T00:25:08.969847+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:08.969847+0000","last_undegraded":"2024-04-26T00:25:08.969847+0000","last_fullsized":"2024-04-26T00:25:08.969847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"20'6","reported_seq":48,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:08.976206+0000","last_change":"2024-04-26T00:25:08.976365+0000","last_active":"2024-04-26T00:25:08.976206+0000","last_peered":"2024-04-26T00:25:08.976206+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:08.976097+0000","last_became_peered":"2024-04-26T00:25:08.976097+0000","last_unstale":"2024-04-26T00:25:08.976206+0000","last_undegraded":"2024-04-26T00:25:08.964101+0000","last_fullsized":"2024-04-26T00:25:08.976206+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973959+0000","last_change":"2024-04-26T00:25:08.973959+0000","last_active":"2024-04-26T00:25:08.973959+0000","last_peered":"2024-04-26T00:25:08.973959+0000","last_clean":"2024-04-26T00:25:08.973959+0000","last_became_active":"2024-04-26T00:25:08.973682+0000","last_became_peered":"2024-04-26T00:25:08.973682+0000","last_unstale":"2024-04-26T00:25:08.973959+0000","last_undegraded":"2024-04-26T00:25:08.973959+0000","last_fullsized":"2024-04-26T00:25:08.973959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:01.958996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"20'3","reported_seq":20,"reported_epoch":29,"state":"active+recovery_unfound","last_fresh":"2024-04-26T00:25:08.971516+0000","last_change":"2024-04-26T00:25:08.971907+0000","last_active":"2024-04-26T00:25:08.971516+0000","last_peered":"2024-04-26T00:25:08.971516+0000","last_clean":"2024-04-26T00:24:44.315138+0000","last_became_active":"2024-04-26T00:25:08.968798+0000","last_became_peered":"2024-04-26T00:25:08.968798+0000","last_unstale":"2024-04-26T00:25:08.971516+0000","last_undegraded":"2024-04-26T00:25:08.966324+0000","last_fullsized":"2024-04-26T00:25:08.971516+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974651+0000","last_change":"2024-04-26T00:25:08.974651+0000","last_active":"2024-04-26T00:25:08.974651+0000","last_peered":"2024-04-26T00:25:08.974651+0000","last_clean":"2024-04-26T00:25:08.974651+0000","last_became_active":"2024-04-26T00:25:08.973555+0000","last_became_peered":"2024-04-26T00:25:08.973555+0000","last_unstale":"2024-04-26T00:25:08.974651+0000","last_undegraded":"2024-04-26T00:25:08.974651+0000","last_fullsized":"2024-04-26T00:25:08.974651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:43.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"20'4","reported_seq":20,"reported_epoch":29,"state":"active+recovery_unfound","last_fresh":"2024-04-26T00:25:08.974084+0000","last_change":"2024-04-26T00:25:08.974681+0000","last_active":"2024-04-26T00:25:08.974084+0000","last_peered":"2024-04-26T00:25:08.974084+0000","last_clean":"2024-04-26T00:24:44.681464+0000","last_became_active":"2024-04-26T00:25:08.971629+0000","last_became_peered":"2024-04-26T00:25:08.971629+0000","last_unstale":"2024-04-26T00:25:08.974084+0000","last_undegraded":"2024-04-26T00:25:08.966794+0000","last_fullsized":"2024-04-26T00:25:08.974084+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"20'1","reported_seq":47,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:08.975924+0000","last_change":"2024-04-26T00:25:08.976154+0000","last_active":"2024-04-26T00:25:08.975924+0000","last_peered":"2024-04-26T00:25:08.975924+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:08.975847+0000","last_became_peered":"2024-04-26T00:25:08.975847+0000","last_unstale":"2024-04-26T00:25:08.975924+0000","last_undegraded":"2024-04-26T00:25:08.963824+0000","last_fullsized":"2024-04-26T00:25:08.975924+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.970682+0000","last_change":"2024-04-26T00:25:08.970682+0000","last_active":"2024-04-26T00:25:08.970682+0000","last_peered":"2024-04-26T00:25:08.970682+0000","last_clean":"2024-04-26T00:25:08.970682+0000","last_became_active":"2024-04-26T00:25:08.970402+0000","last_became_peered":"2024-04-26T00:25:08.970402+0000","last_unstale":"2024-04-26T00:25:08.970682+0000","last_undegraded":"2024-04-26T00:25:08.970682+0000","last_fullsized":"2024-04-26T00:25:08.970682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:30.870253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"20'5","reported_seq":52,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:08.976101+0000","last_change":"2024-04-26T00:25:08.976265+0000","last_active":"2024-04-26T00:25:08.976101+0000","last_peered":"2024-04-26T00:25:08.976101+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:08.976026+0000","last_became_peered":"2024-04-26T00:25:08.976026+0000","last_unstale":"2024-04-26T00:25:08.976101+0000","last_undegraded":"2024-04-26T00:25:08.964098+0000","last_fullsized":"2024-04-26T00:25:08.976101+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.975493+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:08.975493+0000","last_peered":"2024-04-26T00:25:08.975493+0000","last_clean":"2024-04-26T00:25:08.975493+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:08.975493+0000","last_undegraded":"2024-04-26T00:25:08.975493+0000","last_fullsized":"2024-04-26T00:25:08.975493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"20'2","reported_seq":48,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:08.976698+0000","last_change":"2024-04-26T00:25:08.976936+0000","last_active":"2024-04-26T00:25:08.976698+0000","last_peered":"2024-04-26T00:25:08.976698+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:08.976619+0000","last_became_peered":"2024-04-26T00:25:08.976619+0000","last_unstale":"2024-04-26T00:25:08.976698+0000","last_undegraded":"2024-04-26T00:25:08.964511+0000","last_fullsized":"2024-04-26T00:25:08.976698+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974051+0000","last_change":"2024-04-26T00:25:08.974051+0000","last_active":"2024-04-26T00:25:08.974051+0000","last_peered":"2024-04-26T00:25:08.974051+0000","last_clean":"2024-04-26T00:25:08.974051+0000","last_became_active":"2024-04-26T00:25:08.973751+0000","last_became_peered":"2024-04-26T00:25:08.973751+0000","last_unstale":"2024-04-26T00:25:08.974051+0000","last_undegraded":"2024-04-26T00:25:08.974051+0000","last_fullsized":"2024-04-26T00:25:08.974051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:01.153160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974072+0000","last_change":"2024-04-26T00:25:08.974072+0000","last_active":"2024-04-26T00:25:08.974072+0000","last_peered":"2024-04-26T00:25:08.974072+0000","last_clean":"2024-04-26T00:25:08.974072+0000","last_became_active":"2024-04-26T00:25:08.973370+0000","last_became_peered":"2024-04-26T00:25:08.973370+0000","last_unstale":"2024-04-26T00:25:08.974072+0000","last_undegraded":"2024-04-26T00:25:08.974072+0000","last_fullsized":"2024-04-26T00:25:08.974072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:51.793538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"20'9","reported_seq":56,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:08.972567+0000","last_change":"2024-04-26T00:25:08.972679+0000","last_active":"2024-04-26T00:25:08.972567+0000","last_peered":"2024-04-26T00:25:08.972567+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:08.972513+0000","last_became_peered":"2024-04-26T00:25:08.972513+0000","last_unstale":"2024-04-26T00:25:08.972567+0000","last_undegraded":"2024-04-26T00:25:08.963881+0000","last_fullsized":"2024-04-26T00:25:08.972567+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1932,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":13524,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":18354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280519,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27752,"kb_used_data":280,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344088,"statfs":{"total":96636764160,"available":96608346112,"internally_reserved":0,"allocated":286720,"data_stored":80307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28110541},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084292,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27772,"kb_used_data":240,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344068,"statfs":{"total":96636764160,"available":96608325632,"internally_reserved":0,"allocated":245760,"data_stored":75477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":28176080},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":4830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:17.630 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:25:17.974 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:17.975 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/28 objects unfound (96.429%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 53/56 objects degraded (94.643%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":63,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":29,"num_osds":3,"num_up_osds":2,"osd_up_since":1714091107,"num_in_osds":2,"osd_in_since":1714091107,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":10},{"state_name":"active+recovery_wait+degraded","count":8},{"state_name":"active+recovery_unfound","count":5},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":28,"data_bytes":13524,"bytes_used":56856576,"bytes_avail":193216671744,"bytes_total":193273528320,"degraded_objects":53,"degraded_total":56,"degraded_ratio":0.9464285714285714,"unfound_objects":27,"unfound_total":28,"unfound_ratio":0.9642857142857143},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-26T00:25:02.404228+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:25:17.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'fecf292c-395a-497e-b8b4-a892b12f3dd9', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/28 objects unfound (96.429%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 53/56 objects degraded (94.643%), 9 pgs degraded', 'count': 9}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 63, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 29, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1714091107, 'num_in_osds': 2, 'osd_in_since': 1714091107, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovery_wait+degraded', 'count': 8}, {'state_name': 'active+recovery_unfound', 'count': 5}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 28, 'data_bytes': 13524, 'bytes_used': 56856576, 'bytes_avail': 193216671744, 'bytes_total': 193273528320, 'degraded_objects': 53, 'degraded_total': 56, 'degraded_ratio': 0.9464285714285714, 'unfound_objects': 27, 'unfound_total': 28, 'unfound_ratio': 0.9642857142857143}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-04-26T00:25:02.404228+0000', 'services': {'mgr': {'daemons': {'summary': '', 'x': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'mon': {'daemons': {'summary': '', 'a': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'b': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'c': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-04-26T00:25:17.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-04-26T00:25:17.988 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T00:25:17.988 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T00:25:18.119 INFO:teuthology.orchestra.run.smithi072.stdout:94489280521 2024-04-26T00:25:18.119 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:25:18.120 INFO:teuthology.orchestra.run.smithi072.stdout:120259084294 2024-04-26T00:25:18.120 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:25:18.455 INFO:teuthology.orchestra.run.smithi072.stdout:120259084292 2024-04-26T00:25:18.455 INFO:teuthology.orchestra.run.smithi072.stdout:94489280519 2024-04-26T00:25:18.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 94489280521 got 94489280519 for osd.0 2024-04-26T00:25:18.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 120259084294 got 120259084292 for osd.2 2024-04-26T00:25:19.470 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:25:19.471 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:25:19.773 INFO:teuthology.orchestra.run.smithi072.stdout:94489280521 2024-04-26T00:25:19.774 INFO:teuthology.orchestra.run.smithi072.stdout:120259084294 2024-04-26T00:25:19.787 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 94489280521 got 94489280521 for osd.0 2024-04-26T00:25:19.787 DEBUG:teuthology.parallel:result is None 2024-04-26T00:25:19.788 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 120259084294 got 120259084294 for osd.2 2024-04-26T00:25:19.788 DEBUG:teuthology.parallel:result is None 2024-04-26T00:25:19.788 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:25:20.132 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:20.133 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/28 objects unfound (96.429%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 53/56 objects degraded (94.643%), 9 pgs degraded","count":9},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":66,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":29,"num_osds":3,"num_up_osds":2,"osd_up_since":1714091107,"num_in_osds":2,"osd_in_since":1714091107,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":10},{"state_name":"active+recovery_wait+degraded","count":8},{"state_name":"active+recovery_unfound","count":5},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":28,"data_bytes":13524,"bytes_used":56856576,"bytes_avail":193216671744,"bytes_total":193273528320,"degraded_objects":53,"degraded_total":56,"degraded_ratio":0.9464285714285714,"unfound_objects":27,"unfound_total":28,"unfound_ratio":0.9642857142857143},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-26T00:25:02.404228+0000","services":{"mgr":{"daemons":{"summary":"","x":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"mon":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}},"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:25:20.145 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': 'fecf292c-395a-497e-b8b4-a892b12f3dd9', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/28 objects unfound (96.429%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 53/56 objects degraded (94.643%), 9 pgs degraded', 'count': 9}, 'muted': False}}, 'mutes': []}, 'election_epoch': 6, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 66, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 29, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1714091107, 'num_in_osds': 2, 'osd_in_since': 1714091107, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovery_wait+degraded', 'count': 8}, {'state_name': 'active+recovery_unfound', 'count': 5}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 28, 'data_bytes': 13524, 'bytes_used': 56856576, 'bytes_avail': 193216671744, 'bytes_total': 193273528320, 'degraded_objects': 53, 'degraded_total': 56, 'degraded_ratio': 0.9464285714285714, 'unfound_objects': 27, 'unfound_total': 28, 'unfound_ratio': 0.9642857142857143}, 'fsmap': {'epoch': 1, 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-04-26T00:25:02.404228+0000', 'services': {'mgr': {'daemons': {'summary': '', 'x': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'mon': {'daemons': {'summary': '', 'a': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'b': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, 'c': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}, 'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-04-26T00:25:20.145 INFO:tasks.rep_lost_unfound_delete:there are 27 unfound objects 2024-04-26T00:25:20.146 DEBUG:teuthology.orchestra.run.smithi072:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfounddel_pool -t 20 bench 240 write' 2024-04-26T00:25:20.193 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:hints = 1 2024-04-26T00:25:20.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-04-26T00:25:20.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Object prefix: benchmark_data_smithi072_40700 2024-04-26T00:25:20.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:25:20.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 0 0 0 0 0 0 - 0 2024-04-26T00:25:21.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 1 20 2465 2445 9.55027 9.55078 0.0079386 0.00815501 2024-04-26T00:25:22.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 2 20 4767 4747 9.27087 8.99219 0.207997 0.00824399 2024-04-26T00:25:23.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 3 20 6939 6919 9.00848 8.48438 0.00635768 0.00865772 2024-04-26T00:25:24.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 4 20 9520 9500 9.27667 10.082 0.00612634 0.00841183 2024-04-26T00:25:25.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 5 20 12194 12174 9.51023 10.4453 0.00824217 0.00820546 2024-04-26T00:25:26.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 6 20 14682 14662 9.54486 9.71875 0.00739586 0.00817609 2024-04-26T00:25:27.194 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 7 20 16600 16580 9.25153 7.49219 0.00889807 0.0084387 2024-04-26T00:25:28.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 8 20 18941 18921 9.23807 9.14453 0.00894456 0.00844716 2024-04-26T00:25:29.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 9 20 21123 21103 9.15859 8.52344 0.00584223 0.00852527 2024-04-26T00:25:30.149 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:30.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 10 20 23190 23170 9.05009 8.07422 0.0125027 0.00862473 2024-04-26T00:25:30.646 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:30.646 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:30.666 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":61,"stamp":"2024-04-26T00:25:30.410289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":74417364,"num_objects":18193,"num_object_clones":0,"num_object_copies":36386,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":18193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18220,"num_write_kb":72706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18220,"ondisk_log_size":18220,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":259856,"kb_used_data":138236,"kb_used_omap":8,"kb_used_meta":121591,"kb_avail":188483824,"statfs":{"total":193273528320,"available":193007435776,"internally_reserved":0,"allocated":141553664,"data_stored":141176968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":124509597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,2,7,18],"upper_bound":16},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":74403840,"num_objects":18165,"num_object_clones":0,"num_object_copies":36330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18165,"num_write_kb":72660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003014"},"pg_stats":[{"pgid":"2.f","version":"29'1178","reported_seq":1218,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:27.821280+0000","last_change":"2024-04-26T00:25:08.976334+0000","last_active":"2024-04-26T00:25:27.821280+0000","last_peered":"2024-04-26T00:25:27.821280+0000","last_clean":"2024-04-26T00:25:27.821280+0000","last_became_active":"2024-04-26T00:25:08.976010+0000","last_became_peered":"2024-04-26T00:25:08.976010+0000","last_unstale":"2024-04-26T00:25:27.821280+0000","last_undegraded":"2024-04-26T00:25:27.821280+0000","last_fullsized":"2024-04-26T00:25:27.821280+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1178,"log_dups_size":0,"ondisk_log_size":1178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:51.146504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4825088,"num_objects":1178,"num_object_clones":0,"num_object_copies":2356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1178,"num_write_kb":4712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"29'1269","reported_seq":1285,"reported_epoch":29,"state":"active+recovery_unfound+degraded","last_fresh":"2024-04-26T00:25:27.940480+0000","last_change":"2024-04-26T00:25:20.220553+0000","last_active":"2024-04-26T00:25:27.940480+0000","last_peered":"2024-04-26T00:25:27.940480+0000","last_clean":"2024-04-26T00:24:44.453720+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:27.940480+0000","last_undegraded":"2024-04-26T00:25:08.978015+0000","last_fullsized":"2024-04-26T00:25:27.940480+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1269,"log_dups_size":0,"ondisk_log_size":1269,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5182406,"num_objects":1267,"num_object_clones":0,"num_object_copies":2534,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1267,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1269,"num_write_kb":5064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1267}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"29'1132","reported_seq":1144,"reported_epoch":29,"state":"active+recovery_unfound+degraded","last_fresh":"2024-04-26T00:25:27.944239+0000","last_change":"2024-04-26T00:25:20.221656+0000","last_active":"2024-04-26T00:25:27.944239+0000","last_peered":"2024-04-26T00:25:27.944239+0000","last_clean":"2024-04-26T00:24:44.961847+0000","last_became_active":"2024-04-26T00:25:08.973037+0000","last_became_peered":"2024-04-26T00:25:08.973037+0000","last_unstale":"2024-04-26T00:25:27.944239+0000","last_undegraded":"2024-04-26T00:25:08.978413+0000","last_fullsized":"2024-04-26T00:25:27.944239+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4624867,"num_objects":1130,"num_object_clones":0,"num_object_copies":2260,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"29'1125","reported_seq":1144,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:27.933359+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:27.933359+0000","last_peered":"2024-04-26T00:25:27.933359+0000","last_clean":"2024-04-26T00:25:27.933359+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:27.933359+0000","last_undegraded":"2024-04-26T00:25:27.933359+0000","last_fullsized":"2024-04-26T00:25:27.933359+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4608000,"num_objects":1125,"num_object_clones":0,"num_object_copies":2250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1125,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1125,"num_write_kb":4500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"29'1134","reported_seq":1180,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:27.836181+0000","last_change":"2024-04-26T00:25:08.974060+0000","last_active":"2024-04-26T00:25:27.836181+0000","last_peered":"2024-04-26T00:25:27.836181+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:27.836181+0000","last_undegraded":"2024-04-26T00:25:08.963569+0000","last_fullsized":"2024-04-26T00:25:27.836181+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1134,"log_dups_size":0,"ondisk_log_size":1134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4625350,"num_objects":1131,"num_object_clones":0,"num_object_copies":2262,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1134,"num_write_kb":4520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1131}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"29'1131","reported_seq":1177,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:27.821353+0000","last_change":"2024-04-26T00:25:08.972675+0000","last_active":"2024-04-26T00:25:27.821353+0000","last_peered":"2024-04-26T00:25:27.821353+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:08.972454+0000","last_became_peered":"2024-04-26T00:25:08.972454+0000","last_unstale":"2024-04-26T00:25:27.821353+0000","last_undegraded":"2024-04-26T00:25:08.963672+0000","last_fullsized":"2024-04-26T00:25:27.821353+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1131,"log_dups_size":0,"ondisk_log_size":1131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4621254,"num_objects":1130,"num_object_clones":0,"num_object_copies":2260,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1131,"num_write_kb":4515,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1130}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"29'1110","reported_seq":1156,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:27.832966+0000","last_change":"2024-04-26T00:25:08.970817+0000","last_active":"2024-04-26T00:25:27.832966+0000","last_peered":"2024-04-26T00:25:27.832966+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:08.970587+0000","last_became_peered":"2024-04-26T00:25:08.970587+0000","last_unstale":"2024-04-26T00:25:27.832966+0000","last_undegraded":"2024-04-26T00:25:08.964127+0000","last_fullsized":"2024-04-26T00:25:27.832966+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1110,"log_dups_size":0,"ondisk_log_size":1110,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4531142,"num_objects":1108,"num_object_clones":0,"num_object_copies":2216,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1110,"num_write_kb":4428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1108}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"29'1132","reported_seq":1181,"reported_epoch":29,"state":"active+recovering","last_fresh":"2024-04-26T00:25:27.841264+0000","last_change":"2024-04-26T00:25:20.219912+0000","last_active":"2024-04-26T00:25:27.841264+0000","last_peered":"2024-04-26T00:25:27.841264+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:27.841264+0000","last_undegraded":"2024-04-26T00:25:27.841264+0000","last_fullsized":"2024-04-26T00:25:27.841264+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4633059,"num_objects":1132,"num_object_clones":0,"num_object_copies":2264,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973776+0000","last_change":"2024-04-26T00:25:08.973776+0000","last_active":"2024-04-26T00:25:08.973776+0000","last_peered":"2024-04-26T00:25:08.973776+0000","last_clean":"2024-04-26T00:25:08.973776+0000","last_became_active":"2024-04-26T00:25:08.973531+0000","last_became_peered":"2024-04-26T00:25:08.973531+0000","last_unstale":"2024-04-26T00:25:08.973776+0000","last_undegraded":"2024-04-26T00:25:08.973776+0000","last_fullsized":"2024-04-26T00:25:08.973776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:51:51.533562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"29'1112","reported_seq":1122,"reported_epoch":29,"state":"active+recovery_unfound+degraded","last_fresh":"2024-04-26T00:25:27.941019+0000","last_change":"2024-04-26T00:25:20.221584+0000","last_active":"2024-04-26T00:25:27.941019+0000","last_peered":"2024-04-26T00:25:27.941019+0000","last_clean":"2024-04-26T00:24:45.008203+0000","last_became_active":"2024-04-26T00:25:08.972082+0000","last_became_peered":"2024-04-26T00:25:08.972082+0000","last_unstale":"2024-04-26T00:25:27.941019+0000","last_undegraded":"2024-04-26T00:25:08.977167+0000","last_fullsized":"2024-04-26T00:25:27.941019+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1112,"log_dups_size":0,"ondisk_log_size":1112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4535238,"num_objects":1109,"num_object_clones":0,"num_object_copies":2218,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1112,"num_write_kb":4432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1109}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969847+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:08.969847+0000","last_peered":"2024-04-26T00:25:08.969847+0000","last_clean":"2024-04-26T00:25:08.969847+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:08.969847+0000","last_undegraded":"2024-04-26T00:25:08.969847+0000","last_fullsized":"2024-04-26T00:25:08.969847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"29'1164","reported_seq":1206,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:27.832853+0000","last_change":"2024-04-26T00:25:08.976365+0000","last_active":"2024-04-26T00:25:27.832853+0000","last_peered":"2024-04-26T00:25:27.832853+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:08.976097+0000","last_became_peered":"2024-04-26T00:25:08.976097+0000","last_unstale":"2024-04-26T00:25:27.832853+0000","last_undegraded":"2024-04-26T00:25:08.964101+0000","last_fullsized":"2024-04-26T00:25:27.832853+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1164,"log_dups_size":0,"ondisk_log_size":1164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4744134,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1164,"num_write_kb":4636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1160}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973959+0000","last_change":"2024-04-26T00:25:08.973959+0000","last_active":"2024-04-26T00:25:08.973959+0000","last_peered":"2024-04-26T00:25:08.973959+0000","last_clean":"2024-04-26T00:25:08.973959+0000","last_became_active":"2024-04-26T00:25:08.973682+0000","last_became_peered":"2024-04-26T00:25:08.973682+0000","last_unstale":"2024-04-26T00:25:08.973959+0000","last_undegraded":"2024-04-26T00:25:08.973959+0000","last_fullsized":"2024-04-26T00:25:08.973959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:01.958996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"29'1116","reported_seq":1133,"reported_epoch":29,"state":"active+recovery_unfound+degraded","last_fresh":"2024-04-26T00:25:27.945882+0000","last_change":"2024-04-26T00:25:20.221553+0000","last_active":"2024-04-26T00:25:27.945882+0000","last_peered":"2024-04-26T00:25:27.945882+0000","last_clean":"2024-04-26T00:24:44.315138+0000","last_became_active":"2024-04-26T00:25:08.968798+0000","last_became_peered":"2024-04-26T00:25:08.968798+0000","last_unstale":"2024-04-26T00:25:27.945882+0000","last_undegraded":"2024-04-26T00:25:08.971907+0000","last_fullsized":"2024-04-26T00:25:27.945882+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1116,"log_dups_size":0,"ondisk_log_size":1116,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4559814,"num_objects":1115,"num_object_clones":0,"num_object_copies":2230,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1116,"num_write_kb":4455,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1115}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974651+0000","last_change":"2024-04-26T00:25:08.974651+0000","last_active":"2024-04-26T00:25:08.974651+0000","last_peered":"2024-04-26T00:25:08.974651+0000","last_clean":"2024-04-26T00:25:08.974651+0000","last_became_active":"2024-04-26T00:25:08.973555+0000","last_became_peered":"2024-04-26T00:25:08.973555+0000","last_unstale":"2024-04-26T00:25:08.974651+0000","last_undegraded":"2024-04-26T00:25:08.974651+0000","last_fullsized":"2024-04-26T00:25:08.974651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:43.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"29'1108","reported_seq":1124,"reported_epoch":29,"state":"active+recovery_unfound+degraded","last_fresh":"2024-04-26T00:25:27.947811+0000","last_change":"2024-04-26T00:25:20.220384+0000","last_active":"2024-04-26T00:25:27.947811+0000","last_peered":"2024-04-26T00:25:27.947811+0000","last_clean":"2024-04-26T00:24:44.681464+0000","last_became_active":"2024-04-26T00:25:08.971629+0000","last_became_peered":"2024-04-26T00:25:08.971629+0000","last_unstale":"2024-04-26T00:25:27.947811+0000","last_undegraded":"2024-04-26T00:25:08.974681+0000","last_fullsized":"2024-04-26T00:25:27.947811+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1108,"log_dups_size":0,"ondisk_log_size":1108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4522950,"num_objects":1106,"num_object_clones":0,"num_object_copies":2212,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1108,"num_write_kb":4419,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1106}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"29'1132","reported_seq":1178,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:27.840107+0000","last_change":"2024-04-26T00:25:08.976154+0000","last_active":"2024-04-26T00:25:27.840107+0000","last_peered":"2024-04-26T00:25:27.840107+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:08.975847+0000","last_became_peered":"2024-04-26T00:25:08.975847+0000","last_unstale":"2024-04-26T00:25:27.840107+0000","last_undegraded":"2024-04-26T00:25:08.963824+0000","last_fullsized":"2024-04-26T00:25:27.840107+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1132,"log_dups_size":0,"ondisk_log_size":1132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4633059,"num_objects":1132,"num_object_clones":0,"num_object_copies":2264,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1132,"num_write_kb":4525,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1132}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.970682+0000","last_change":"2024-04-26T00:25:08.970682+0000","last_active":"2024-04-26T00:25:08.970682+0000","last_peered":"2024-04-26T00:25:08.970682+0000","last_clean":"2024-04-26T00:25:08.970682+0000","last_became_active":"2024-04-26T00:25:08.970402+0000","last_became_peered":"2024-04-26T00:25:08.970402+0000","last_unstale":"2024-04-26T00:25:08.970682+0000","last_undegraded":"2024-04-26T00:25:08.970682+0000","last_fullsized":"2024-04-26T00:25:08.970682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:30.870253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"29'1074","reported_seq":1120,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:27.844417+0000","last_change":"2024-04-26T00:25:08.976265+0000","last_active":"2024-04-26T00:25:27.844417+0000","last_peered":"2024-04-26T00:25:27.844417+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:08.976026+0000","last_became_peered":"2024-04-26T00:25:08.976026+0000","last_unstale":"2024-04-26T00:25:27.844417+0000","last_undegraded":"2024-04-26T00:25:08.964098+0000","last_fullsized":"2024-04-26T00:25:27.844417+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1074,"log_dups_size":0,"ondisk_log_size":1074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4380073,"num_objects":1072,"num_object_clones":0,"num_object_copies":2144,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1074,"num_write_kb":4280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.975493+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:08.975493+0000","last_peered":"2024-04-26T00:25:08.975493+0000","last_clean":"2024-04-26T00:25:08.975493+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:08.975493+0000","last_undegraded":"2024-04-26T00:25:08.975493+0000","last_fullsized":"2024-04-26T00:25:08.975493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"29'1153","reported_seq":1199,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:27.844658+0000","last_change":"2024-04-26T00:25:08.976936+0000","last_active":"2024-04-26T00:25:27.844658+0000","last_peered":"2024-04-26T00:25:27.844658+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:08.976619+0000","last_became_peered":"2024-04-26T00:25:08.976619+0000","last_unstale":"2024-04-26T00:25:27.844658+0000","last_undegraded":"2024-04-26T00:25:08.964511+0000","last_fullsized":"2024-04-26T00:25:27.844658+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1153,"log_dups_size":0,"ondisk_log_size":1153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4715462,"num_objects":1153,"num_object_clones":0,"num_object_copies":2306,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1153,"num_write_kb":4606,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1153}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974051+0000","last_change":"2024-04-26T00:25:08.974051+0000","last_active":"2024-04-26T00:25:08.974051+0000","last_peered":"2024-04-26T00:25:08.974051+0000","last_clean":"2024-04-26T00:25:08.974051+0000","last_became_active":"2024-04-26T00:25:08.973751+0000","last_became_peered":"2024-04-26T00:25:08.973751+0000","last_unstale":"2024-04-26T00:25:08.974051+0000","last_undegraded":"2024-04-26T00:25:08.974051+0000","last_fullsized":"2024-04-26T00:25:08.974051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:01.153160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974072+0000","last_change":"2024-04-26T00:25:08.974072+0000","last_active":"2024-04-26T00:25:08.974072+0000","last_peered":"2024-04-26T00:25:08.974072+0000","last_clean":"2024-04-26T00:25:08.974072+0000","last_became_active":"2024-04-26T00:25:08.973370+0000","last_became_peered":"2024-04-26T00:25:08.973370+0000","last_unstale":"2024-04-26T00:25:08.974072+0000","last_undegraded":"2024-04-26T00:25:08.974072+0000","last_fullsized":"2024-04-26T00:25:08.974072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:51.793538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"29'1150","reported_seq":1197,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:27.837045+0000","last_change":"2024-04-26T00:25:08.972679+0000","last_active":"2024-04-26T00:25:27.837045+0000","last_peered":"2024-04-26T00:25:27.837045+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:08.972513+0000","last_became_peered":"2024-04-26T00:25:08.972513+0000","last_unstale":"2024-04-26T00:25:27.837045+0000","last_undegraded":"2024-04-26T00:25:08.963881+0000","last_fullsized":"2024-04-26T00:25:27.837045+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":1150,"log_dups_size":0,"ondisk_log_size":1150,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4675468,"num_objects":1145,"num_object_clones":0,"num_object_copies":2290,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1150,"num_write_kb":4571,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1144}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":74417364,"num_objects":18193,"num_object_clones":0,"num_object_copies":36386,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":53,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":18193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18220,"num_write_kb":72706,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":149336064,"data_stored":149198770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18220,"ondisk_log_size":18220,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280523,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":122728,"kb_used_data":64736,"kb_used_omap":4,"kb_used_meta":57979,"kb_avail":94249112,"statfs":{"total":96636764160,"available":96511090688,"internally_reserved":0,"allocated":66289664,"data_stored":66083251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":59371213},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,3,8],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084296,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":137128,"kb_used_data":73500,"kb_used_omap":4,"kb_used_meta":63611,"kb_avail":94234712,"statfs":{"total":96636764160,"available":96496345088,"internally_reserved":0,"allocated":75264000,"data_stored":75093717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":65138384},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,10],"upper_bound":16},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":74125312,"data_stored":74089182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75096064,"data_stored":75096064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:30.668 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.f 2024-04-26T00:25:30.668 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-04-26T00:25:30.668 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-04-26T00:25:30.859 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:30.859 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:30.859 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:30.859 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:30.859 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:30.859 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_4", 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 1224592654, 2024-04-26T00:25:30.860 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'3", 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'1", 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_6", 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:30.861 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 3101443614, 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'4", 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'2", 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "NotRecovering", 2024-04-26T00:25:30.862 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:30.863 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:30.863 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:30.863 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:30.863 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:30.863 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:30.863 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:30.863 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:30.863 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:30.873 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.e on osd.2 2024-04-26T00:25:30.873 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost delete 2024-04-26T00:25:31.081 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:31.100 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-04-26T00:25:31.100 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-04-26T00:25:31.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 11 20 25008 24988 8.8729 7.10156 0.00823897 0.00879645 2024-04-26T00:25:31.295 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:31.295 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:31.295 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 1, 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 1, 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_7", 2024-04-26T00:25:31.296 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 2358579165, 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'3", 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:31.297 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "NotRecovering", 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:31.298 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:31.313 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.d on osd.2 2024-04-26T00:25:31.313 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost delete 2024-04-26T00:25:31.526 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 1 objects unfound and apparently lost marking 2024-04-26T00:25:31.550 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.c 2024-04-26T00:25:31.550 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-04-26T00:25:31.550 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-04-26T00:25:31.744 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:31.745 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_3", 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 943960171, 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'5", 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:31.746 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_1", 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 2885884443, 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'4", 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'1", 2024-04-26T00:25:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:31.748 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:31.766 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.b on osd.0 2024-04-26T00:25:31.766 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost delete 2024-04-26T00:25:31.978 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:31.997 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-04-26T00:25:31.997 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-04-26T00:25:32.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 12 20 26806 26786 8.71873 7.02344 0.00908567 0.00895274 2024-04-26T00:25:32.196 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:32.196 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:32.197 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_9", 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 1217469130, 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'3", 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'1", 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:32.198 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_9", 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 170713514, 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'2", 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:32.199 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:32.200 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:32.220 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.a on osd.0 2024-04-26T00:25:32.220 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost delete 2024-04-26T00:25:32.516 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:32.531 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-04-26T00:25:32.531 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-04-26T00:25:32.735 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:32.735 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:32.735 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:32.735 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:32.735 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:32.735 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:32.735 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_5", 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 2998549577, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:32.736 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'4", 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'2", 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_2", 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 4032093161, 2024-04-26T00:25:32.737 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'3", 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'1", 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:32.738 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:32.739 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:32.739 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:32.739 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:32.739 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:32.750 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.9 on osd.0 2024-04-26T00:25:32.750 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost delete 2024-04-26T00:25:32.972 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:32.988 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.8 state active+recovering 2024-04-26T00:25:32.989 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-04-26T00:25:33.182 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:33.182 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:33.182 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:33.182 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 1, 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 1, 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_8", 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:33.183 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 2730327960, 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'1", 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:33.184 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:33.185 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:33.185 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:33.185 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:33.185 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:33.185 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:33.185 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:33.185 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:33.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 13 20 28324 28304 8.50415 5.92969 0.00798994 0.00917973 2024-04-26T00:25:33.202 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.8 on osd.0 2024-04-26T00:25:33.202 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost delete 2024-04-26T00:25:33.423 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 1 objects unfound and apparently lost marking 2024-04-26T00:25:33.440 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.0 2024-04-26T00:25:33.440 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-04-26T00:25:33.441 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-04-26T00:25:33.632 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:33.632 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:33.632 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:33.632 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:33.632 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:33.632 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:33.632 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_8", 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 1901787523, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:33.633 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'5", 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'3", 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_1", 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 3122379363, 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:33.634 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'4", 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "NotRecovering", 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:33.635 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:33.636 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:33.636 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:33.636 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:33.636 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:33.651 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.3 on osd.2 2024-04-26T00:25:33.651 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost delete 2024-04-26T00:25:33.856 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:33.875 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.1 2024-04-26T00:25:33.875 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-04-26T00:25:33.875 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-04-26T00:25:34.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 14 20 29668 29648 8.27168 5.25 0.0108203 0.00943731 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:34.226 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_6", 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 1708673474, 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:34.227 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'5", 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_9", 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 831402034, 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:34.228 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'6", 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:34.229 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:34.230 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:34.230 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:34.246 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.2 on osd.0 2024-04-26T00:25:34.246 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost delete 2024-04-26T00:25:34.458 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:34.475 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.2 2024-04-26T00:25:34.475 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-04-26T00:25:34.475 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-04-26T00:25:34.669 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:34.670 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_3", 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 459768081, 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'3", 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'1", 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:34.671 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_3", 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 480221105, 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:34.672 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'2", 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "NotRecovering", 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:34.673 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:34.674 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:34.693 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.1 on osd.2 2024-04-26T00:25:34.693 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost delete 2024-04-26T00:25:34.915 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:34.935 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.3 2024-04-26T00:25:34.935 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-04-26T00:25:34.935 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-04-26T00:25:35.056 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:25:35.075+0000 7ffb0da16640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 3 pgs recovery_unfound (PG_DAMAGED) 2024-04-26T00:25:35.133 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:35.133 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:35.133 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:35.133 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_5", 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:35.134 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 4248945696, 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'3", 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:35.135 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_5", 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 4212789968, 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'4", 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:35.136 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "NotRecovering", 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:35.137 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:35.153 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.0 on osd.2 2024-04-26T00:25:35.154 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost delete 2024-04-26T00:25:35.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 15 20 31356 31336 8.15977 6.59375 0.00903043 0.00956801 2024-04-26T00:25:35.366 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:35.383 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-04-26T00:25:35.383 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-04-26T00:25:35.619 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:35.619 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:35.619 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:35.619 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:35.619 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:35.619 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:35.619 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:35.619 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 1, 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 1, 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_7", 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 3761017527, 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:35.620 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'1", 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:35.621 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:35.622 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:35.622 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:35.634 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.7 on osd.0 2024-04-26T00:25:35.635 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost delete 2024-04-26T00:25:35.847 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 1 objects unfound and apparently lost marking 2024-04-26T00:25:35.868 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.4 2024-04-26T00:25:35.868 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-04-26T00:25:35.869 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:36.082 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 3, 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 3, 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_8", 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 1861424966, 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.083 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'5", 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_6", 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 3910131350, 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.084 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'4", 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_4", 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 2787839990, 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:36.085 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'3", 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:36.086 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:36.087 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:36.087 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:36.087 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:36.102 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.6 on osd.0 2024-04-26T00:25:36.102 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost delete 2024-04-26T00:25:36.195 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 16 20 32954 32934 8.03989 6.24219 0.0138193 0.0097103 2024-04-26T00:25:36.323 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 3 objects unfound and apparently lost marking 2024-04-26T00:25:36.342 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.5 2024-04-26T00:25:36.343 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-04-26T00:25:36.343 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-04-26T00:25:36.535 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 2, 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 2, 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:36.536 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_1", 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 26281365, 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'1", 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "new_2", 2024-04-26T00:25:36.537 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 3196340981, 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'2", 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:36.538 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:36.539 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:36.539 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:36.552 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.5 on osd.0 2024-04-26T00:25:36.552 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost delete 2024-04-26T00:25:36.986 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 2 objects unfound and apparently lost marking 2024-04-26T00:25:37.005 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.6 2024-04-26T00:25:37.005 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.7 2024-04-26T00:25:37.005 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.4 state active+recovery_wait+degraded 2024-04-26T00:25:37.005 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-04-26T00:25:37.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 17 20 34194 34174 7.85186 4.84375 0.00754534 0.009944 2024-04-26T00:25:37.205 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:37.205 INFO:teuthology.orchestra.run.smithi072.stdout: "offset": { 2024-04-26T00:25:37.205 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "", 2024-04-26T00:25:37.205 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:37.205 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": 0, 2024-04-26T00:25:37.205 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 0, 2024-04-26T00:25:37.205 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": -9223372036854775808, 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "num_missing": 4, 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "num_unfound": 3, 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "objects": [ 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_4", 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 873891716, 2024-04-26T00:25:37.206 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'8", 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existing_7", 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:37.207 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 3996155460, 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'9", 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "18'6", 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [(483, 18446744073709551132)], clean_omap: true, new_object: false", 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": { 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "oid": "existed_2", 2024-04-26T00:25:37.208 INFO:teuthology.orchestra.run.smithi072.stdout: "key": "", 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "snapid": -2, 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "hash": 1767667508, 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 0, 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "pool": 2, 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "namespace": "" 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "need": "20'7", 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "have": "0'0", 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "flags": "none", 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "locations": [] 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "state": "Active", 2024-04-26T00:25:37.209 INFO:teuthology.orchestra.run.smithi072.stdout: "available_might_have_unfound": true, 2024-04-26T00:25:37.210 INFO:teuthology.orchestra.run.smithi072.stdout: "might_have_unfound": [ 2024-04-26T00:25:37.210 INFO:teuthology.orchestra.run.smithi072.stdout: { 2024-04-26T00:25:37.210 INFO:teuthology.orchestra.run.smithi072.stdout: "osd": "1", 2024-04-26T00:25:37.210 INFO:teuthology.orchestra.run.smithi072.stdout: "status": "osd is down" 2024-04-26T00:25:37.210 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-04-26T00:25:37.210 INFO:teuthology.orchestra.run.smithi072.stdout: ], 2024-04-26T00:25:37.210 INFO:teuthology.orchestra.run.smithi072.stdout: "more": false 2024-04-26T00:25:37.210 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:37.227 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.4 on osd.0 2024-04-26T00:25:37.227 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost delete 2024-04-26T00:25:37.448 INFO:teuthology.orchestra.run.smithi072.stderr:pg has 3 objects unfound and apparently lost marking 2024-04-26T00:25:37.464 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-04-26T00:25:37.665 INFO:teuthology.orchestra.run.smithi072.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-04-26T00:25:37.683 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-04-26T00:25:37.885 INFO:teuthology.orchestra.run.smithi072.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-04-26T00:25:37.904 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-26T00:25:37.904 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-26T00:25:38.131 INFO:teuthology.orchestra.run.smithi072.stdout:94489280526 2024-04-26T00:25:38.131 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:25:38.138 INFO:teuthology.orchestra.run.smithi072.stdout:120259084299 2024-04-26T00:25:38.138 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:25:38.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 18 20 35743 35723 7.75177 6.05078 0.015947 0.01007 2024-04-26T00:25:38.537 INFO:teuthology.orchestra.run.smithi072.stdout:94489280524 2024-04-26T00:25:38.549 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 94489280526 got 94489280524 for osd.0 2024-04-26T00:25:38.565 INFO:teuthology.orchestra.run.smithi072.stdout:120259084297 2024-04-26T00:25:38.577 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 120259084299 got 120259084297 for osd.2 2024-04-26T00:25:39.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 19 20 37037 37017 7.6098 5.05469 0.00965872 0.0102587 2024-04-26T00:25:39.549 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-26T00:25:39.577 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-26T00:25:40.073 INFO:teuthology.orchestra.run.smithi072.stdout:94489280526 2024-04-26T00:25:40.096 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 94489280526 got 94489280526 for osd.0 2024-04-26T00:25:40.096 DEBUG:teuthology.parallel:result is None 2024-04-26T00:25:40.125 INFO:teuthology.orchestra.run.smithi072.stdout:120259084299 2024-04-26T00:25:40.149 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 120259084299 got 120259084299 for osd.2 2024-04-26T00:25:40.150 DEBUG:teuthology.parallel:result is None 2024-04-26T00:25:40.150 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-04-26T00:25:40.150 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:40.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:25:40.216813+0000 min lat: 0.00244021 max lat: 0.241933 avg lat: 0.0104714 2024-04-26T00:25:40.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:25:40.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 20 20 38198 38178 7.45605 4.53516 0.00855862 0.0104714 2024-04-26T00:25:40.704 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:40.704 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:40.723 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-26T00:25:40.412564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145884372,"num_objects":35641,"num_object_clones":0,"num_object_copies":71282,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35668,"num_write_kb":142498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35695,"ondisk_log_size":35695,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":446612,"kb_used_data":283332,"kb_used_omap":8,"kb_used_meta":163255,"kb_avail":188297068,"statfs":{"total":193273528320,"available":192816197632,"internally_reserved":0,"allocated":290131968,"data_stored":289769724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":167173533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,9,5,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":71467008,"num_objects":17448,"num_object_clones":0,"num_object_copies":34896,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":17448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17448,"num_write_kb":69792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002966"},"pg_stats":[{"pgid":"2.f","version":"29'2246","reported_seq":2286,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.116531+0000","last_change":"2024-04-26T00:25:08.976334+0000","last_active":"2024-04-26T00:25:38.116531+0000","last_peered":"2024-04-26T00:25:38.116531+0000","last_clean":"2024-04-26T00:25:38.116531+0000","last_became_active":"2024-04-26T00:25:08.976010+0000","last_became_peered":"2024-04-26T00:25:08.976010+0000","last_unstale":"2024-04-26T00:25:38.116531+0000","last_undegraded":"2024-04-26T00:25:38.116531+0000","last_fullsized":"2024-04-26T00:25:38.116531+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2246,"log_dups_size":0,"ondisk_log_size":2246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:51.146504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9199616,"num_objects":2246,"num_object_clones":0,"num_object_copies":4492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2246,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"29'2377","reported_seq":2403,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.097320+0000","last_change":"2024-04-26T00:25:31.147930+0000","last_active":"2024-04-26T00:25:38.097320+0000","last_peered":"2024-04-26T00:25:38.097320+0000","last_clean":"2024-04-26T00:25:38.097320+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:38.097320+0000","last_undegraded":"2024-04-26T00:25:38.097320+0000","last_fullsized":"2024-04-26T00:25:38.097320+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.973931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"29'2228","reported_seq":2248,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.109223+0000","last_change":"2024-04-26T00:25:31.575878+0000","last_active":"2024-04-26T00:25:38.109223+0000","last_peered":"2024-04-26T00:25:38.109223+0000","last_clean":"2024-04-26T00:25:38.109223+0000","last_became_active":"2024-04-26T00:25:08.973037+0000","last_became_peered":"2024-04-26T00:25:08.973037+0000","last_unstale":"2024-04-26T00:25:38.109223+0000","last_undegraded":"2024-04-26T00:25:38.109223+0000","last_fullsized":"2024-04-26T00:25:38.109223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2228,"log_dups_size":0,"ondisk_log_size":2228,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.974688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109987,"num_objects":2225,"num_object_clones":0,"num_object_copies":4450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2227,"num_write_kb":8898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"29'2167","reported_seq":2186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.102972+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:38.102972+0000","last_peered":"2024-04-26T00:25:38.102972+0000","last_clean":"2024-04-26T00:25:38.102972+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:38.102972+0000","last_undegraded":"2024-04-26T00:25:38.102972+0000","last_fullsized":"2024-04-26T00:25:38.102972+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2167,"log_dups_size":0,"ondisk_log_size":2167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8876032,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2167,"num_write_kb":8668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"29'2263","reported_seq":2307,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.118533+0000","last_change":"2024-04-26T00:25:08.974060+0000","last_active":"2024-04-26T00:25:38.118533+0000","last_peered":"2024-04-26T00:25:38.118533+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:38.118533+0000","last_undegraded":"2024-04-26T00:25:08.963569+0000","last_fullsized":"2024-04-26T00:25:38.118533+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9241542,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2261,"num_write_kb":9028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"29'2231","reported_seq":2275,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.118839+0000","last_change":"2024-04-26T00:25:08.972675+0000","last_active":"2024-04-26T00:25:38.118839+0000","last_peered":"2024-04-26T00:25:38.118839+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:08.972454+0000","last_became_peered":"2024-04-26T00:25:08.972454+0000","last_unstale":"2024-04-26T00:25:38.118839+0000","last_undegraded":"2024-04-26T00:25:08.963672+0000","last_fullsized":"2024-04-26T00:25:38.118839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9118662,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":8907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"29'2223","reported_seq":2269,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.122787+0000","last_change":"2024-04-26T00:25:08.970817+0000","last_active":"2024-04-26T00:25:38.122787+0000","last_peered":"2024-04-26T00:25:38.122787+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:08.970587+0000","last_became_peered":"2024-04-26T00:25:08.970587+0000","last_unstale":"2024-04-26T00:25:38.122787+0000","last_undegraded":"2024-04-26T00:25:08.964127+0000","last_fullsized":"2024-04-26T00:25:38.122787+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2223,"log_dups_size":0,"ondisk_log_size":2223,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9081798,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2221,"num_write_kb":8872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"29'2184","reported_seq":2233,"reported_epoch":29,"state":"active+recovering","last_fresh":"2024-04-26T00:25:38.118380+0000","last_change":"2024-04-26T00:25:20.219912+0000","last_active":"2024-04-26T00:25:38.118380+0000","last_peered":"2024-04-26T00:25:38.118380+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:38.118380+0000","last_undegraded":"2024-04-26T00:25:38.118380+0000","last_fullsized":"2024-04-26T00:25:38.118380+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937955,"num_objects":2183,"num_object_clones":0,"num_object_copies":4366,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2183,"num_write_kb":8729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2183}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973776+0000","last_change":"2024-04-26T00:25:08.973776+0000","last_active":"2024-04-26T00:25:08.973776+0000","last_peered":"2024-04-26T00:25:08.973776+0000","last_clean":"2024-04-26T00:25:08.973776+0000","last_became_active":"2024-04-26T00:25:08.973531+0000","last_became_peered":"2024-04-26T00:25:08.973531+0000","last_unstale":"2024-04-26T00:25:08.973776+0000","last_undegraded":"2024-04-26T00:25:08.973776+0000","last_fullsized":"2024-04-26T00:25:08.973776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:51:51.533562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"29'2157","reported_seq":2176,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.133355+0000","last_change":"2024-04-26T00:25:33.924747+0000","last_active":"2024-04-26T00:25:38.133355+0000","last_peered":"2024-04-26T00:25:38.133355+0000","last_clean":"2024-04-26T00:25:38.133355+0000","last_became_active":"2024-04-26T00:25:08.972082+0000","last_became_peered":"2024-04-26T00:25:08.972082+0000","last_unstale":"2024-04-26T00:25:38.133355+0000","last_undegraded":"2024-04-26T00:25:38.133355+0000","last_fullsized":"2024-04-26T00:25:38.133355+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2157,"log_dups_size":0,"ondisk_log_size":2157,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:39.889295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8807366,"num_objects":2152,"num_object_clones":0,"num_object_copies":4304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2155,"num_write_kb":8604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969847+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:08.969847+0000","last_peered":"2024-04-26T00:25:08.969847+0000","last_clean":"2024-04-26T00:25:08.969847+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:08.969847+0000","last_undegraded":"2024-04-26T00:25:08.969847+0000","last_fullsized":"2024-04-26T00:25:08.969847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"29'2226","reported_seq":2267,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.077141+0000","last_change":"2024-04-26T00:25:08.976365+0000","last_active":"2024-04-26T00:25:38.077141+0000","last_peered":"2024-04-26T00:25:38.077141+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:08.976097+0000","last_became_peered":"2024-04-26T00:25:08.976097+0000","last_unstale":"2024-04-26T00:25:38.077141+0000","last_undegraded":"2024-04-26T00:25:08.964101+0000","last_fullsized":"2024-04-26T00:25:38.077141+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2226,"log_dups_size":0,"ondisk_log_size":2226,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9085894,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2224,"num_write_kb":8876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2220}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973959+0000","last_change":"2024-04-26T00:25:08.973959+0000","last_active":"2024-04-26T00:25:08.973959+0000","last_peered":"2024-04-26T00:25:08.973959+0000","last_clean":"2024-04-26T00:25:08.973959+0000","last_became_active":"2024-04-26T00:25:08.973682+0000","last_became_peered":"2024-04-26T00:25:08.973682+0000","last_unstale":"2024-04-26T00:25:08.973959+0000","last_undegraded":"2024-04-26T00:25:08.973959+0000","last_fullsized":"2024-04-26T00:25:08.973959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:01.958996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"29'2259","reported_seq":2284,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.138849+0000","last_change":"2024-04-26T00:25:34.995156+0000","last_active":"2024-04-26T00:25:38.138849+0000","last_peered":"2024-04-26T00:25:38.138849+0000","last_clean":"2024-04-26T00:25:38.138849+0000","last_became_active":"2024-04-26T00:25:08.968798+0000","last_became_peered":"2024-04-26T00:25:08.968798+0000","last_unstale":"2024-04-26T00:25:38.138849+0000","last_undegraded":"2024-04-26T00:25:38.138849+0000","last_fullsized":"2024-04-26T00:25:38.138849+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2259,"log_dups_size":0,"ondisk_log_size":2259,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:40.844729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9233350,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2257,"num_write_kb":9019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974651+0000","last_change":"2024-04-26T00:25:08.974651+0000","last_active":"2024-04-26T00:25:08.974651+0000","last_peered":"2024-04-26T00:25:08.974651+0000","last_clean":"2024-04-26T00:25:08.974651+0000","last_became_active":"2024-04-26T00:25:08.973555+0000","last_became_peered":"2024-04-26T00:25:08.973555+0000","last_unstale":"2024-04-26T00:25:08.974651+0000","last_undegraded":"2024-04-26T00:25:08.974651+0000","last_fullsized":"2024-04-26T00:25:08.974651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:43.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"29'2188","reported_seq":2214,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.097096+0000","last_change":"2024-04-26T00:25:35.448359+0000","last_active":"2024-04-26T00:25:38.097096+0000","last_peered":"2024-04-26T00:25:38.097096+0000","last_clean":"2024-04-26T00:25:38.097096+0000","last_became_active":"2024-04-26T00:25:08.971629+0000","last_became_peered":"2024-04-26T00:25:08.971629+0000","last_unstale":"2024-04-26T00:25:38.097096+0000","last_undegraded":"2024-04-26T00:25:38.097096+0000","last_fullsized":"2024-04-26T00:25:38.097096+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2188,"log_dups_size":0,"ondisk_log_size":2188,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:40.844748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8938438,"num_objects":2184,"num_object_clones":0,"num_object_copies":4368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2186,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"29'2250","reported_seq":2296,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.122018+0000","last_change":"2024-04-26T00:25:08.976154+0000","last_active":"2024-04-26T00:25:38.122018+0000","last_peered":"2024-04-26T00:25:38.122018+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:08.975847+0000","last_became_peered":"2024-04-26T00:25:08.975847+0000","last_unstale":"2024-04-26T00:25:38.122018+0000","last_undegraded":"2024-04-26T00:25:08.963824+0000","last_fullsized":"2024-04-26T00:25:38.122018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9208291,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2249,"num_write_kb":8993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.970682+0000","last_change":"2024-04-26T00:25:08.970682+0000","last_active":"2024-04-26T00:25:08.970682+0000","last_peered":"2024-04-26T00:25:08.970682+0000","last_clean":"2024-04-26T00:25:08.970682+0000","last_became_active":"2024-04-26T00:25:08.970402+0000","last_became_peered":"2024-04-26T00:25:08.970402+0000","last_unstale":"2024-04-26T00:25:08.970682+0000","last_undegraded":"2024-04-26T00:25:08.970682+0000","last_fullsized":"2024-04-26T00:25:08.970682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:30.870253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"29'2169","reported_seq":2214,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.119697+0000","last_change":"2024-04-26T00:25:08.976265+0000","last_active":"2024-04-26T00:25:38.119697+0000","last_peered":"2024-04-26T00:25:38.119697+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:08.976026+0000","last_became_peered":"2024-04-26T00:25:08.976026+0000","last_unstale":"2024-04-26T00:25:38.119697+0000","last_undegraded":"2024-04-26T00:25:08.964098+0000","last_fullsized":"2024-04-26T00:25:38.119697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2169,"log_dups_size":0,"ondisk_log_size":2169,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8852905,"num_objects":2164,"num_object_clones":0,"num_object_copies":4328,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2166,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.975493+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:08.975493+0000","last_peered":"2024-04-26T00:25:08.975493+0000","last_clean":"2024-04-26T00:25:08.975493+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:08.975493+0000","last_undegraded":"2024-04-26T00:25:08.975493+0000","last_fullsized":"2024-04-26T00:25:08.975493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"29'2242","reported_seq":2286,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.106865+0000","last_change":"2024-04-26T00:25:08.976936+0000","last_active":"2024-04-26T00:25:38.106865+0000","last_peered":"2024-04-26T00:25:38.106865+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:08.976619+0000","last_became_peered":"2024-04-26T00:25:08.976619+0000","last_unstale":"2024-04-26T00:25:38.106865+0000","last_undegraded":"2024-04-26T00:25:08.964511+0000","last_fullsized":"2024-04-26T00:25:38.106865+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9167814,"num_objects":2240,"num_object_clones":0,"num_object_copies":4480,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2240,"num_write_kb":8954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974051+0000","last_change":"2024-04-26T00:25:08.974051+0000","last_active":"2024-04-26T00:25:08.974051+0000","last_peered":"2024-04-26T00:25:08.974051+0000","last_clean":"2024-04-26T00:25:08.974051+0000","last_became_active":"2024-04-26T00:25:08.973751+0000","last_became_peered":"2024-04-26T00:25:08.973751+0000","last_unstale":"2024-04-26T00:25:08.974051+0000","last_undegraded":"2024-04-26T00:25:08.974051+0000","last_fullsized":"2024-04-26T00:25:08.974051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:01.153160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974072+0000","last_change":"2024-04-26T00:25:08.974072+0000","last_active":"2024-04-26T00:25:08.974072+0000","last_peered":"2024-04-26T00:25:08.974072+0000","last_clean":"2024-04-26T00:25:08.974072+0000","last_became_active":"2024-04-26T00:25:08.973370+0000","last_became_peered":"2024-04-26T00:25:08.973370+0000","last_unstale":"2024-04-26T00:25:08.974072+0000","last_undegraded":"2024-04-26T00:25:08.974072+0000","last_fullsized":"2024-04-26T00:25:08.974072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:51.793538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"29'2285","reported_seq":2330,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.074654+0000","last_change":"2024-04-26T00:25:08.972679+0000","last_active":"2024-04-26T00:25:38.074654+0000","last_peered":"2024-04-26T00:25:38.074654+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:08.972513+0000","last_became_peered":"2024-04-26T00:25:08.972513+0000","last_unstale":"2024-04-26T00:25:38.074654+0000","last_undegraded":"2024-04-26T00:25:08.963881+0000","last_fullsized":"2024-04-26T00:25:38.074654+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9312140,"num_objects":2277,"num_object_clones":0,"num_object_copies":4554,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2282,"num_write_kb":9099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145884372,"num_objects":35641,"num_object_clones":0,"num_object_copies":71282,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35668,"num_write_kb":142498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":291942400,"data_stored":291819558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35695,"ondisk_log_size":35695,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280526,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223096,"kb_used_data":141488,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94148744,"statfs":{"total":96636764160,"available":96408313856,"internally_reserved":0,"allocated":144883712,"data_stored":144691751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":83553997},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,4,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084299,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223516,"kb_used_data":141844,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94148324,"statfs":{"total":96636764160,"available":96407883776,"internally_reserved":0,"allocated":145248256,"data_stored":145077973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":83619536},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,4,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":145907712,"data_stored":145886034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":145920000,"data_stored":145920000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:40.725 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:41.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 21 20 39702 39682 7.38074 5.875 0.0122609 0.010578 2024-04-26T00:25:41.265 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:41.266 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:41.286 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-26T00:25:40.412564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145884372,"num_objects":35641,"num_object_clones":0,"num_object_copies":71282,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35668,"num_write_kb":142498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35695,"ondisk_log_size":35695,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":446612,"kb_used_data":283332,"kb_used_omap":8,"kb_used_meta":163255,"kb_avail":188297068,"statfs":{"total":193273528320,"available":192816197632,"internally_reserved":0,"allocated":290131968,"data_stored":289769724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":167173533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,9,5,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":71467008,"num_objects":17448,"num_object_clones":0,"num_object_copies":34896,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":17448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17448,"num_write_kb":69792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002966"},"pg_stats":[{"pgid":"2.f","version":"29'2246","reported_seq":2286,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.116531+0000","last_change":"2024-04-26T00:25:08.976334+0000","last_active":"2024-04-26T00:25:38.116531+0000","last_peered":"2024-04-26T00:25:38.116531+0000","last_clean":"2024-04-26T00:25:38.116531+0000","last_became_active":"2024-04-26T00:25:08.976010+0000","last_became_peered":"2024-04-26T00:25:08.976010+0000","last_unstale":"2024-04-26T00:25:38.116531+0000","last_undegraded":"2024-04-26T00:25:38.116531+0000","last_fullsized":"2024-04-26T00:25:38.116531+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2246,"log_dups_size":0,"ondisk_log_size":2246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:51.146504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9199616,"num_objects":2246,"num_object_clones":0,"num_object_copies":4492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2246,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"29'2377","reported_seq":2403,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.097320+0000","last_change":"2024-04-26T00:25:31.147930+0000","last_active":"2024-04-26T00:25:38.097320+0000","last_peered":"2024-04-26T00:25:38.097320+0000","last_clean":"2024-04-26T00:25:38.097320+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:38.097320+0000","last_undegraded":"2024-04-26T00:25:38.097320+0000","last_fullsized":"2024-04-26T00:25:38.097320+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.973931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"29'2228","reported_seq":2248,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.109223+0000","last_change":"2024-04-26T00:25:31.575878+0000","last_active":"2024-04-26T00:25:38.109223+0000","last_peered":"2024-04-26T00:25:38.109223+0000","last_clean":"2024-04-26T00:25:38.109223+0000","last_became_active":"2024-04-26T00:25:08.973037+0000","last_became_peered":"2024-04-26T00:25:08.973037+0000","last_unstale":"2024-04-26T00:25:38.109223+0000","last_undegraded":"2024-04-26T00:25:38.109223+0000","last_fullsized":"2024-04-26T00:25:38.109223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2228,"log_dups_size":0,"ondisk_log_size":2228,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.974688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109987,"num_objects":2225,"num_object_clones":0,"num_object_copies":4450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2227,"num_write_kb":8898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"29'2167","reported_seq":2186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.102972+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:38.102972+0000","last_peered":"2024-04-26T00:25:38.102972+0000","last_clean":"2024-04-26T00:25:38.102972+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:38.102972+0000","last_undegraded":"2024-04-26T00:25:38.102972+0000","last_fullsized":"2024-04-26T00:25:38.102972+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2167,"log_dups_size":0,"ondisk_log_size":2167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8876032,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2167,"num_write_kb":8668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"29'2263","reported_seq":2307,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.118533+0000","last_change":"2024-04-26T00:25:08.974060+0000","last_active":"2024-04-26T00:25:38.118533+0000","last_peered":"2024-04-26T00:25:38.118533+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:38.118533+0000","last_undegraded":"2024-04-26T00:25:08.963569+0000","last_fullsized":"2024-04-26T00:25:38.118533+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9241542,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2261,"num_write_kb":9028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"29'2231","reported_seq":2275,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.118839+0000","last_change":"2024-04-26T00:25:08.972675+0000","last_active":"2024-04-26T00:25:38.118839+0000","last_peered":"2024-04-26T00:25:38.118839+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:08.972454+0000","last_became_peered":"2024-04-26T00:25:08.972454+0000","last_unstale":"2024-04-26T00:25:38.118839+0000","last_undegraded":"2024-04-26T00:25:08.963672+0000","last_fullsized":"2024-04-26T00:25:38.118839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9118662,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":8907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"29'2223","reported_seq":2269,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.122787+0000","last_change":"2024-04-26T00:25:08.970817+0000","last_active":"2024-04-26T00:25:38.122787+0000","last_peered":"2024-04-26T00:25:38.122787+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:08.970587+0000","last_became_peered":"2024-04-26T00:25:08.970587+0000","last_unstale":"2024-04-26T00:25:38.122787+0000","last_undegraded":"2024-04-26T00:25:08.964127+0000","last_fullsized":"2024-04-26T00:25:38.122787+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2223,"log_dups_size":0,"ondisk_log_size":2223,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9081798,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2221,"num_write_kb":8872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"29'2184","reported_seq":2233,"reported_epoch":29,"state":"active+recovering","last_fresh":"2024-04-26T00:25:38.118380+0000","last_change":"2024-04-26T00:25:20.219912+0000","last_active":"2024-04-26T00:25:38.118380+0000","last_peered":"2024-04-26T00:25:38.118380+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:38.118380+0000","last_undegraded":"2024-04-26T00:25:38.118380+0000","last_fullsized":"2024-04-26T00:25:38.118380+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937955,"num_objects":2183,"num_object_clones":0,"num_object_copies":4366,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2183,"num_write_kb":8729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2183}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973776+0000","last_change":"2024-04-26T00:25:08.973776+0000","last_active":"2024-04-26T00:25:08.973776+0000","last_peered":"2024-04-26T00:25:08.973776+0000","last_clean":"2024-04-26T00:25:08.973776+0000","last_became_active":"2024-04-26T00:25:08.973531+0000","last_became_peered":"2024-04-26T00:25:08.973531+0000","last_unstale":"2024-04-26T00:25:08.973776+0000","last_undegraded":"2024-04-26T00:25:08.973776+0000","last_fullsized":"2024-04-26T00:25:08.973776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:51:51.533562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"29'2157","reported_seq":2176,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.133355+0000","last_change":"2024-04-26T00:25:33.924747+0000","last_active":"2024-04-26T00:25:38.133355+0000","last_peered":"2024-04-26T00:25:38.133355+0000","last_clean":"2024-04-26T00:25:38.133355+0000","last_became_active":"2024-04-26T00:25:08.972082+0000","last_became_peered":"2024-04-26T00:25:08.972082+0000","last_unstale":"2024-04-26T00:25:38.133355+0000","last_undegraded":"2024-04-26T00:25:38.133355+0000","last_fullsized":"2024-04-26T00:25:38.133355+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2157,"log_dups_size":0,"ondisk_log_size":2157,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:39.889295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8807366,"num_objects":2152,"num_object_clones":0,"num_object_copies":4304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2155,"num_write_kb":8604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969847+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:08.969847+0000","last_peered":"2024-04-26T00:25:08.969847+0000","last_clean":"2024-04-26T00:25:08.969847+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:08.969847+0000","last_undegraded":"2024-04-26T00:25:08.969847+0000","last_fullsized":"2024-04-26T00:25:08.969847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"29'2226","reported_seq":2267,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.077141+0000","last_change":"2024-04-26T00:25:08.976365+0000","last_active":"2024-04-26T00:25:38.077141+0000","last_peered":"2024-04-26T00:25:38.077141+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:08.976097+0000","last_became_peered":"2024-04-26T00:25:08.976097+0000","last_unstale":"2024-04-26T00:25:38.077141+0000","last_undegraded":"2024-04-26T00:25:08.964101+0000","last_fullsized":"2024-04-26T00:25:38.077141+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2226,"log_dups_size":0,"ondisk_log_size":2226,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9085894,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2224,"num_write_kb":8876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2220}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973959+0000","last_change":"2024-04-26T00:25:08.973959+0000","last_active":"2024-04-26T00:25:08.973959+0000","last_peered":"2024-04-26T00:25:08.973959+0000","last_clean":"2024-04-26T00:25:08.973959+0000","last_became_active":"2024-04-26T00:25:08.973682+0000","last_became_peered":"2024-04-26T00:25:08.973682+0000","last_unstale":"2024-04-26T00:25:08.973959+0000","last_undegraded":"2024-04-26T00:25:08.973959+0000","last_fullsized":"2024-04-26T00:25:08.973959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:01.958996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"29'2259","reported_seq":2284,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.138849+0000","last_change":"2024-04-26T00:25:34.995156+0000","last_active":"2024-04-26T00:25:38.138849+0000","last_peered":"2024-04-26T00:25:38.138849+0000","last_clean":"2024-04-26T00:25:38.138849+0000","last_became_active":"2024-04-26T00:25:08.968798+0000","last_became_peered":"2024-04-26T00:25:08.968798+0000","last_unstale":"2024-04-26T00:25:38.138849+0000","last_undegraded":"2024-04-26T00:25:38.138849+0000","last_fullsized":"2024-04-26T00:25:38.138849+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2259,"log_dups_size":0,"ondisk_log_size":2259,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:40.844729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9233350,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2257,"num_write_kb":9019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974651+0000","last_change":"2024-04-26T00:25:08.974651+0000","last_active":"2024-04-26T00:25:08.974651+0000","last_peered":"2024-04-26T00:25:08.974651+0000","last_clean":"2024-04-26T00:25:08.974651+0000","last_became_active":"2024-04-26T00:25:08.973555+0000","last_became_peered":"2024-04-26T00:25:08.973555+0000","last_unstale":"2024-04-26T00:25:08.974651+0000","last_undegraded":"2024-04-26T00:25:08.974651+0000","last_fullsized":"2024-04-26T00:25:08.974651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:43.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"29'2188","reported_seq":2214,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.097096+0000","last_change":"2024-04-26T00:25:35.448359+0000","last_active":"2024-04-26T00:25:38.097096+0000","last_peered":"2024-04-26T00:25:38.097096+0000","last_clean":"2024-04-26T00:25:38.097096+0000","last_became_active":"2024-04-26T00:25:08.971629+0000","last_became_peered":"2024-04-26T00:25:08.971629+0000","last_unstale":"2024-04-26T00:25:38.097096+0000","last_undegraded":"2024-04-26T00:25:38.097096+0000","last_fullsized":"2024-04-26T00:25:38.097096+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2188,"log_dups_size":0,"ondisk_log_size":2188,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:40.844748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8938438,"num_objects":2184,"num_object_clones":0,"num_object_copies":4368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2186,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"29'2250","reported_seq":2296,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.122018+0000","last_change":"2024-04-26T00:25:08.976154+0000","last_active":"2024-04-26T00:25:38.122018+0000","last_peered":"2024-04-26T00:25:38.122018+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:08.975847+0000","last_became_peered":"2024-04-26T00:25:08.975847+0000","last_unstale":"2024-04-26T00:25:38.122018+0000","last_undegraded":"2024-04-26T00:25:08.963824+0000","last_fullsized":"2024-04-26T00:25:38.122018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9208291,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2249,"num_write_kb":8993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.970682+0000","last_change":"2024-04-26T00:25:08.970682+0000","last_active":"2024-04-26T00:25:08.970682+0000","last_peered":"2024-04-26T00:25:08.970682+0000","last_clean":"2024-04-26T00:25:08.970682+0000","last_became_active":"2024-04-26T00:25:08.970402+0000","last_became_peered":"2024-04-26T00:25:08.970402+0000","last_unstale":"2024-04-26T00:25:08.970682+0000","last_undegraded":"2024-04-26T00:25:08.970682+0000","last_fullsized":"2024-04-26T00:25:08.970682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:30.870253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"29'2169","reported_seq":2214,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.119697+0000","last_change":"2024-04-26T00:25:08.976265+0000","last_active":"2024-04-26T00:25:38.119697+0000","last_peered":"2024-04-26T00:25:38.119697+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:08.976026+0000","last_became_peered":"2024-04-26T00:25:08.976026+0000","last_unstale":"2024-04-26T00:25:38.119697+0000","last_undegraded":"2024-04-26T00:25:08.964098+0000","last_fullsized":"2024-04-26T00:25:38.119697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2169,"log_dups_size":0,"ondisk_log_size":2169,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8852905,"num_objects":2164,"num_object_clones":0,"num_object_copies":4328,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2166,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.975493+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:08.975493+0000","last_peered":"2024-04-26T00:25:08.975493+0000","last_clean":"2024-04-26T00:25:08.975493+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:08.975493+0000","last_undegraded":"2024-04-26T00:25:08.975493+0000","last_fullsized":"2024-04-26T00:25:08.975493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"29'2242","reported_seq":2286,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.106865+0000","last_change":"2024-04-26T00:25:08.976936+0000","last_active":"2024-04-26T00:25:38.106865+0000","last_peered":"2024-04-26T00:25:38.106865+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:08.976619+0000","last_became_peered":"2024-04-26T00:25:08.976619+0000","last_unstale":"2024-04-26T00:25:38.106865+0000","last_undegraded":"2024-04-26T00:25:08.964511+0000","last_fullsized":"2024-04-26T00:25:38.106865+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9167814,"num_objects":2240,"num_object_clones":0,"num_object_copies":4480,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2240,"num_write_kb":8954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974051+0000","last_change":"2024-04-26T00:25:08.974051+0000","last_active":"2024-04-26T00:25:08.974051+0000","last_peered":"2024-04-26T00:25:08.974051+0000","last_clean":"2024-04-26T00:25:08.974051+0000","last_became_active":"2024-04-26T00:25:08.973751+0000","last_became_peered":"2024-04-26T00:25:08.973751+0000","last_unstale":"2024-04-26T00:25:08.974051+0000","last_undegraded":"2024-04-26T00:25:08.974051+0000","last_fullsized":"2024-04-26T00:25:08.974051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:01.153160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974072+0000","last_change":"2024-04-26T00:25:08.974072+0000","last_active":"2024-04-26T00:25:08.974072+0000","last_peered":"2024-04-26T00:25:08.974072+0000","last_clean":"2024-04-26T00:25:08.974072+0000","last_became_active":"2024-04-26T00:25:08.973370+0000","last_became_peered":"2024-04-26T00:25:08.973370+0000","last_unstale":"2024-04-26T00:25:08.974072+0000","last_undegraded":"2024-04-26T00:25:08.974072+0000","last_fullsized":"2024-04-26T00:25:08.974072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:51.793538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"29'2285","reported_seq":2330,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.074654+0000","last_change":"2024-04-26T00:25:08.972679+0000","last_active":"2024-04-26T00:25:38.074654+0000","last_peered":"2024-04-26T00:25:38.074654+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:08.972513+0000","last_became_peered":"2024-04-26T00:25:08.972513+0000","last_unstale":"2024-04-26T00:25:38.074654+0000","last_undegraded":"2024-04-26T00:25:08.963881+0000","last_fullsized":"2024-04-26T00:25:38.074654+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9312140,"num_objects":2277,"num_object_clones":0,"num_object_copies":4554,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2282,"num_write_kb":9099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145884372,"num_objects":35641,"num_object_clones":0,"num_object_copies":71282,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35668,"num_write_kb":142498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":291942400,"data_stored":291819558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35695,"ondisk_log_size":35695,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280526,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223096,"kb_used_data":141488,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94148744,"statfs":{"total":96636764160,"available":96408313856,"internally_reserved":0,"allocated":144883712,"data_stored":144691751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":83553997},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,4,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084299,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223516,"kb_used_data":141844,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94148324,"statfs":{"total":96636764160,"available":96407883776,"internally_reserved":0,"allocated":145248256,"data_stored":145077973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":83619536},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,4,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":145907712,"data_stored":145886034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":145920000,"data_stored":145920000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:41.289 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:41.984 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:41.985 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:42.003 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-04-26T00:25:40.412564+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":145884372,"num_objects":35641,"num_object_clones":0,"num_object_copies":71282,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35668,"num_write_kb":142498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35695,"ondisk_log_size":35695,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":446612,"kb_used_data":283332,"kb_used_omap":8,"kb_used_meta":163255,"kb_avail":188297068,"statfs":{"total":193273528320,"available":192816197632,"internally_reserved":0,"allocated":290131968,"data_stored":289769724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":167173533},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,0,9,5,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":71467008,"num_objects":17448,"num_object_clones":0,"num_object_copies":34896,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":17448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17448,"num_write_kb":69792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002966"},"pg_stats":[{"pgid":"2.f","version":"29'2246","reported_seq":2286,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.116531+0000","last_change":"2024-04-26T00:25:08.976334+0000","last_active":"2024-04-26T00:25:38.116531+0000","last_peered":"2024-04-26T00:25:38.116531+0000","last_clean":"2024-04-26T00:25:38.116531+0000","last_became_active":"2024-04-26T00:25:08.976010+0000","last_became_peered":"2024-04-26T00:25:08.976010+0000","last_unstale":"2024-04-26T00:25:38.116531+0000","last_undegraded":"2024-04-26T00:25:38.116531+0000","last_fullsized":"2024-04-26T00:25:38.116531+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2246,"log_dups_size":0,"ondisk_log_size":2246,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:51.146504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9199616,"num_objects":2246,"num_object_clones":0,"num_object_copies":4492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2246,"num_write_kb":8984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"29'2377","reported_seq":2403,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.097320+0000","last_change":"2024-04-26T00:25:31.147930+0000","last_active":"2024-04-26T00:25:38.097320+0000","last_peered":"2024-04-26T00:25:38.097320+0000","last_clean":"2024-04-26T00:25:38.097320+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:38.097320+0000","last_undegraded":"2024-04-26T00:25:38.097320+0000","last_fullsized":"2024-04-26T00:25:38.097320+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2377,"log_dups_size":0,"ondisk_log_size":2377,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.973931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9712582,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2375,"num_write_kb":9488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"29'2228","reported_seq":2248,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.109223+0000","last_change":"2024-04-26T00:25:31.575878+0000","last_active":"2024-04-26T00:25:38.109223+0000","last_peered":"2024-04-26T00:25:38.109223+0000","last_clean":"2024-04-26T00:25:38.109223+0000","last_became_active":"2024-04-26T00:25:08.973037+0000","last_became_peered":"2024-04-26T00:25:08.973037+0000","last_unstale":"2024-04-26T00:25:38.109223+0000","last_undegraded":"2024-04-26T00:25:38.109223+0000","last_fullsized":"2024-04-26T00:25:38.109223+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2228,"log_dups_size":0,"ondisk_log_size":2228,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.974688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109987,"num_objects":2225,"num_object_clones":0,"num_object_copies":4450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2227,"num_write_kb":8898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"29'2167","reported_seq":2186,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.102972+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:38.102972+0000","last_peered":"2024-04-26T00:25:38.102972+0000","last_clean":"2024-04-26T00:25:38.102972+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:38.102972+0000","last_undegraded":"2024-04-26T00:25:38.102972+0000","last_fullsized":"2024-04-26T00:25:38.102972+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2167,"log_dups_size":0,"ondisk_log_size":2167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8876032,"num_objects":2167,"num_object_clones":0,"num_object_copies":4334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2167,"num_write_kb":8668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"29'2263","reported_seq":2307,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.118533+0000","last_change":"2024-04-26T00:25:08.974060+0000","last_active":"2024-04-26T00:25:38.118533+0000","last_peered":"2024-04-26T00:25:38.118533+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:38.118533+0000","last_undegraded":"2024-04-26T00:25:08.963569+0000","last_fullsized":"2024-04-26T00:25:38.118533+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2263,"log_dups_size":0,"ondisk_log_size":2263,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9241542,"num_objects":2258,"num_object_clones":0,"num_object_copies":4516,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2261,"num_write_kb":9028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2258}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"29'2231","reported_seq":2275,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.118839+0000","last_change":"2024-04-26T00:25:08.972675+0000","last_active":"2024-04-26T00:25:38.118839+0000","last_peered":"2024-04-26T00:25:38.118839+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:08.972454+0000","last_became_peered":"2024-04-26T00:25:08.972454+0000","last_unstale":"2024-04-26T00:25:38.118839+0000","last_undegraded":"2024-04-26T00:25:08.963672+0000","last_fullsized":"2024-04-26T00:25:38.118839+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2231,"log_dups_size":0,"ondisk_log_size":2231,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9118662,"num_objects":2228,"num_object_clones":0,"num_object_copies":4456,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2229,"num_write_kb":8907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2228}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"29'2223","reported_seq":2269,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.122787+0000","last_change":"2024-04-26T00:25:08.970817+0000","last_active":"2024-04-26T00:25:38.122787+0000","last_peered":"2024-04-26T00:25:38.122787+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:08.970587+0000","last_became_peered":"2024-04-26T00:25:08.970587+0000","last_unstale":"2024-04-26T00:25:38.122787+0000","last_undegraded":"2024-04-26T00:25:08.964127+0000","last_fullsized":"2024-04-26T00:25:38.122787+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2223,"log_dups_size":0,"ondisk_log_size":2223,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9081798,"num_objects":2219,"num_object_clones":0,"num_object_copies":4438,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2221,"num_write_kb":8872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"29'2184","reported_seq":2233,"reported_epoch":29,"state":"active+recovering","last_fresh":"2024-04-26T00:25:38.118380+0000","last_change":"2024-04-26T00:25:20.219912+0000","last_active":"2024-04-26T00:25:38.118380+0000","last_peered":"2024-04-26T00:25:38.118380+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:38.118380+0000","last_undegraded":"2024-04-26T00:25:38.118380+0000","last_fullsized":"2024-04-26T00:25:38.118380+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2184,"log_dups_size":0,"ondisk_log_size":2184,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8937955,"num_objects":2183,"num_object_clones":0,"num_object_copies":4366,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2183,"num_write_kb":8729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2183}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973776+0000","last_change":"2024-04-26T00:25:08.973776+0000","last_active":"2024-04-26T00:25:08.973776+0000","last_peered":"2024-04-26T00:25:08.973776+0000","last_clean":"2024-04-26T00:25:08.973776+0000","last_became_active":"2024-04-26T00:25:08.973531+0000","last_became_peered":"2024-04-26T00:25:08.973531+0000","last_unstale":"2024-04-26T00:25:08.973776+0000","last_undegraded":"2024-04-26T00:25:08.973776+0000","last_fullsized":"2024-04-26T00:25:08.973776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:51:51.533562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"29'2157","reported_seq":2176,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.133355+0000","last_change":"2024-04-26T00:25:33.924747+0000","last_active":"2024-04-26T00:25:38.133355+0000","last_peered":"2024-04-26T00:25:38.133355+0000","last_clean":"2024-04-26T00:25:38.133355+0000","last_became_active":"2024-04-26T00:25:08.972082+0000","last_became_peered":"2024-04-26T00:25:08.972082+0000","last_unstale":"2024-04-26T00:25:38.133355+0000","last_undegraded":"2024-04-26T00:25:38.133355+0000","last_fullsized":"2024-04-26T00:25:38.133355+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2157,"log_dups_size":0,"ondisk_log_size":2157,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:39.889295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8807366,"num_objects":2152,"num_object_clones":0,"num_object_copies":4304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2155,"num_write_kb":8604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969847+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:08.969847+0000","last_peered":"2024-04-26T00:25:08.969847+0000","last_clean":"2024-04-26T00:25:08.969847+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:08.969847+0000","last_undegraded":"2024-04-26T00:25:08.969847+0000","last_fullsized":"2024-04-26T00:25:08.969847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"29'2226","reported_seq":2267,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.077141+0000","last_change":"2024-04-26T00:25:08.976365+0000","last_active":"2024-04-26T00:25:38.077141+0000","last_peered":"2024-04-26T00:25:38.077141+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:08.976097+0000","last_became_peered":"2024-04-26T00:25:08.976097+0000","last_unstale":"2024-04-26T00:25:38.077141+0000","last_undegraded":"2024-04-26T00:25:08.964101+0000","last_fullsized":"2024-04-26T00:25:38.077141+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2226,"log_dups_size":0,"ondisk_log_size":2226,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9085894,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2224,"num_write_kb":8876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2220}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973959+0000","last_change":"2024-04-26T00:25:08.973959+0000","last_active":"2024-04-26T00:25:08.973959+0000","last_peered":"2024-04-26T00:25:08.973959+0000","last_clean":"2024-04-26T00:25:08.973959+0000","last_became_active":"2024-04-26T00:25:08.973682+0000","last_became_peered":"2024-04-26T00:25:08.973682+0000","last_unstale":"2024-04-26T00:25:08.973959+0000","last_undegraded":"2024-04-26T00:25:08.973959+0000","last_fullsized":"2024-04-26T00:25:08.973959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:01.958996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"29'2259","reported_seq":2284,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.138849+0000","last_change":"2024-04-26T00:25:34.995156+0000","last_active":"2024-04-26T00:25:38.138849+0000","last_peered":"2024-04-26T00:25:38.138849+0000","last_clean":"2024-04-26T00:25:38.138849+0000","last_became_active":"2024-04-26T00:25:08.968798+0000","last_became_peered":"2024-04-26T00:25:08.968798+0000","last_unstale":"2024-04-26T00:25:38.138849+0000","last_undegraded":"2024-04-26T00:25:38.138849+0000","last_fullsized":"2024-04-26T00:25:38.138849+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2259,"log_dups_size":0,"ondisk_log_size":2259,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:40.844729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9233350,"num_objects":2256,"num_object_clones":0,"num_object_copies":4512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2257,"num_write_kb":9019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974651+0000","last_change":"2024-04-26T00:25:08.974651+0000","last_active":"2024-04-26T00:25:08.974651+0000","last_peered":"2024-04-26T00:25:08.974651+0000","last_clean":"2024-04-26T00:25:08.974651+0000","last_became_active":"2024-04-26T00:25:08.973555+0000","last_became_peered":"2024-04-26T00:25:08.973555+0000","last_unstale":"2024-04-26T00:25:08.974651+0000","last_undegraded":"2024-04-26T00:25:08.974651+0000","last_fullsized":"2024-04-26T00:25:08.974651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:43.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"29'2188","reported_seq":2214,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:38.097096+0000","last_change":"2024-04-26T00:25:35.448359+0000","last_active":"2024-04-26T00:25:38.097096+0000","last_peered":"2024-04-26T00:25:38.097096+0000","last_clean":"2024-04-26T00:25:38.097096+0000","last_became_active":"2024-04-26T00:25:08.971629+0000","last_became_peered":"2024-04-26T00:25:08.971629+0000","last_unstale":"2024-04-26T00:25:38.097096+0000","last_undegraded":"2024-04-26T00:25:38.097096+0000","last_fullsized":"2024-04-26T00:25:38.097096+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2188,"log_dups_size":0,"ondisk_log_size":2188,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:40.844748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8938438,"num_objects":2184,"num_object_clones":0,"num_object_copies":4368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2186,"num_write_kb":8731,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"29'2250","reported_seq":2296,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.122018+0000","last_change":"2024-04-26T00:25:08.976154+0000","last_active":"2024-04-26T00:25:38.122018+0000","last_peered":"2024-04-26T00:25:38.122018+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:08.975847+0000","last_became_peered":"2024-04-26T00:25:08.975847+0000","last_unstale":"2024-04-26T00:25:38.122018+0000","last_undegraded":"2024-04-26T00:25:08.963824+0000","last_fullsized":"2024-04-26T00:25:38.122018+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2250,"log_dups_size":0,"ondisk_log_size":2250,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9208291,"num_objects":2249,"num_object_clones":0,"num_object_copies":4498,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2249,"num_write_kb":8993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2249}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.970682+0000","last_change":"2024-04-26T00:25:08.970682+0000","last_active":"2024-04-26T00:25:08.970682+0000","last_peered":"2024-04-26T00:25:08.970682+0000","last_clean":"2024-04-26T00:25:08.970682+0000","last_became_active":"2024-04-26T00:25:08.970402+0000","last_became_peered":"2024-04-26T00:25:08.970402+0000","last_unstale":"2024-04-26T00:25:08.970682+0000","last_undegraded":"2024-04-26T00:25:08.970682+0000","last_fullsized":"2024-04-26T00:25:08.970682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:30.870253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"29'2169","reported_seq":2214,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.119697+0000","last_change":"2024-04-26T00:25:08.976265+0000","last_active":"2024-04-26T00:25:38.119697+0000","last_peered":"2024-04-26T00:25:38.119697+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:08.976026+0000","last_became_peered":"2024-04-26T00:25:08.976026+0000","last_unstale":"2024-04-26T00:25:38.119697+0000","last_undegraded":"2024-04-26T00:25:08.964098+0000","last_fullsized":"2024-04-26T00:25:38.119697+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2169,"log_dups_size":0,"ondisk_log_size":2169,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8852905,"num_objects":2164,"num_object_clones":0,"num_object_copies":4328,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2166,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2164}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.975493+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:08.975493+0000","last_peered":"2024-04-26T00:25:08.975493+0000","last_clean":"2024-04-26T00:25:08.975493+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:08.975493+0000","last_undegraded":"2024-04-26T00:25:08.975493+0000","last_fullsized":"2024-04-26T00:25:08.975493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"29'2242","reported_seq":2286,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.106865+0000","last_change":"2024-04-26T00:25:08.976936+0000","last_active":"2024-04-26T00:25:38.106865+0000","last_peered":"2024-04-26T00:25:38.106865+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:08.976619+0000","last_became_peered":"2024-04-26T00:25:08.976619+0000","last_unstale":"2024-04-26T00:25:38.106865+0000","last_undegraded":"2024-04-26T00:25:08.964511+0000","last_fullsized":"2024-04-26T00:25:38.106865+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2242,"log_dups_size":0,"ondisk_log_size":2242,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9167814,"num_objects":2240,"num_object_clones":0,"num_object_copies":4480,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2240,"num_write_kb":8954,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974051+0000","last_change":"2024-04-26T00:25:08.974051+0000","last_active":"2024-04-26T00:25:08.974051+0000","last_peered":"2024-04-26T00:25:08.974051+0000","last_clean":"2024-04-26T00:25:08.974051+0000","last_became_active":"2024-04-26T00:25:08.973751+0000","last_became_peered":"2024-04-26T00:25:08.973751+0000","last_unstale":"2024-04-26T00:25:08.974051+0000","last_undegraded":"2024-04-26T00:25:08.974051+0000","last_fullsized":"2024-04-26T00:25:08.974051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:01.153160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974072+0000","last_change":"2024-04-26T00:25:08.974072+0000","last_active":"2024-04-26T00:25:08.974072+0000","last_peered":"2024-04-26T00:25:08.974072+0000","last_clean":"2024-04-26T00:25:08.974072+0000","last_became_active":"2024-04-26T00:25:08.973370+0000","last_became_peered":"2024-04-26T00:25:08.973370+0000","last_unstale":"2024-04-26T00:25:08.974072+0000","last_undegraded":"2024-04-26T00:25:08.974072+0000","last_fullsized":"2024-04-26T00:25:08.974072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:51.793538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"29'2285","reported_seq":2330,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:38.074654+0000","last_change":"2024-04-26T00:25:08.972679+0000","last_active":"2024-04-26T00:25:38.074654+0000","last_peered":"2024-04-26T00:25:38.074654+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:08.972513+0000","last_became_peered":"2024-04-26T00:25:08.972513+0000","last_unstale":"2024-04-26T00:25:38.074654+0000","last_undegraded":"2024-04-26T00:25:08.963881+0000","last_fullsized":"2024-04-26T00:25:38.074654+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2285,"log_dups_size":0,"ondisk_log_size":2285,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9312140,"num_objects":2277,"num_object_clones":0,"num_object_copies":4554,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2282,"num_write_kb":9099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2276}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":145884372,"num_objects":35641,"num_object_clones":0,"num_object_copies":71282,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35668,"num_write_kb":142498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":291942400,"data_stored":291819558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35695,"ondisk_log_size":35695,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280526,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223096,"kb_used_data":141488,"kb_used_omap":4,"kb_used_meta":81595,"kb_avail":94148744,"statfs":{"total":96636764160,"available":96408313856,"internally_reserved":0,"allocated":144883712,"data_stored":144691751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":83553997},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,4,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084299,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":223516,"kb_used_data":141844,"kb_used_omap":4,"kb_used_meta":81659,"kb_avail":94148324,"statfs":{"total":96636764160,"available":96407883776,"internally_reserved":0,"allocated":145248256,"data_stored":145077973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":83619536},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,4,1,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":145907712,"data_stored":145886034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":145920000,"data_stored":145920000,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:42.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 22 20 40932 40912 7.26363 4.80469 0.0119346 0.0107493 2024-04-26T00:25:43.196 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 23 20 41913 41893 7.11442 3.83203 0.0130915 0.0109747 2024-04-26T00:25:44.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 24 20 43459 43439 7.06959 6.03906 0.0149185 0.0110435 2024-04-26T00:25:45.005 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:45.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 25 20 44732 44712 6.98567 4.97266 0.0173825 0.0111752 2024-04-26T00:25:45.527 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:45.529 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:45.548 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-26T00:25:44.413271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":170747092,"num_objects":41711,"num_object_clones":0,"num_object_copies":83422,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41738,"num_write_kb":166778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41765,"ondisk_log_size":41765,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":550092,"kb_used_data":334068,"kb_used_omap":8,"kb_used_meta":215991,"kb_avail":188193588,"statfs":{"total":193273528320,"available":192710234112,"internally_reserved":0,"allocated":342085632,"data_stored":341723388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":221175197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,11,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":96329728,"num_objects":23518,"num_object_clones":0,"num_object_copies":47036,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":23518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23518,"num_write_kb":94072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002489"},"pg_stats":[{"pgid":"2.f","version":"29'2633","reported_seq":2673,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.850956+0000","last_change":"2024-04-26T00:25:08.976334+0000","last_active":"2024-04-26T00:25:42.850956+0000","last_peered":"2024-04-26T00:25:42.850956+0000","last_clean":"2024-04-26T00:25:42.850956+0000","last_became_active":"2024-04-26T00:25:08.976010+0000","last_became_peered":"2024-04-26T00:25:08.976010+0000","last_unstale":"2024-04-26T00:25:42.850956+0000","last_undegraded":"2024-04-26T00:25:42.850956+0000","last_fullsized":"2024-04-26T00:25:42.850956+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2633,"log_dups_size":0,"ondisk_log_size":2633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:51.146504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10784768,"num_objects":2633,"num_object_clones":0,"num_object_copies":5266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2633,"num_write_kb":10532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"29'2785","reported_seq":2811,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.895772+0000","last_change":"2024-04-26T00:25:31.147930+0000","last_active":"2024-04-26T00:25:42.895772+0000","last_peered":"2024-04-26T00:25:42.895772+0000","last_clean":"2024-04-26T00:25:42.895772+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:42.895772+0000","last_undegraded":"2024-04-26T00:25:42.895772+0000","last_fullsized":"2024-04-26T00:25:42.895772+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2785,"log_dups_size":0,"ondisk_log_size":2785,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.973931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383750,"num_objects":2781,"num_object_clones":0,"num_object_copies":5562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2783,"num_write_kb":11120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"29'2620","reported_seq":2640,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.847913+0000","last_change":"2024-04-26T00:25:31.575878+0000","last_active":"2024-04-26T00:25:42.847913+0000","last_peered":"2024-04-26T00:25:42.847913+0000","last_clean":"2024-04-26T00:25:42.847913+0000","last_became_active":"2024-04-26T00:25:08.973037+0000","last_became_peered":"2024-04-26T00:25:08.973037+0000","last_unstale":"2024-04-26T00:25:42.847913+0000","last_undegraded":"2024-04-26T00:25:42.847913+0000","last_fullsized":"2024-04-26T00:25:42.847913+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2620,"log_dups_size":0,"ondisk_log_size":2620,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.974688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10715619,"num_objects":2617,"num_object_clones":0,"num_object_copies":5234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2619,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"29'2573","reported_seq":2592,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.897671+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:42.897671+0000","last_peered":"2024-04-26T00:25:42.897671+0000","last_clean":"2024-04-26T00:25:42.897671+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:42.897671+0000","last_undegraded":"2024-04-26T00:25:42.897671+0000","last_fullsized":"2024-04-26T00:25:42.897671+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10539008,"num_objects":2573,"num_object_clones":0,"num_object_copies":5146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2573,"num_write_kb":10292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"29'2669","reported_seq":2714,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.786855+0000","last_change":"2024-04-26T00:25:08.974060+0000","last_active":"2024-04-26T00:25:42.786855+0000","last_peered":"2024-04-26T00:25:42.786855+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:42.786855+0000","last_undegraded":"2024-04-26T00:25:08.963569+0000","last_fullsized":"2024-04-26T00:25:42.786855+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2669,"log_dups_size":0,"ondisk_log_size":2669,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10904518,"num_objects":2664,"num_object_clones":0,"num_object_copies":5328,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2667,"num_write_kb":10652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2664}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"29'2592","reported_seq":2638,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.862424+0000","last_change":"2024-04-26T00:25:08.972675+0000","last_active":"2024-04-26T00:25:42.862424+0000","last_peered":"2024-04-26T00:25:42.862424+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:08.972454+0000","last_became_peered":"2024-04-26T00:25:08.972454+0000","last_unstale":"2024-04-26T00:25:42.862424+0000","last_undegraded":"2024-04-26T00:25:08.963672+0000","last_fullsized":"2024-04-26T00:25:42.862424+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2592,"log_dups_size":0,"ondisk_log_size":2592,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10597318,"num_objects":2589,"num_object_clones":0,"num_object_copies":5178,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"29'2592","reported_seq":2637,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.858792+0000","last_change":"2024-04-26T00:25:08.970817+0000","last_active":"2024-04-26T00:25:42.858792+0000","last_peered":"2024-04-26T00:25:42.858792+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:08.970587+0000","last_became_peered":"2024-04-26T00:25:08.970587+0000","last_unstale":"2024-04-26T00:25:42.858792+0000","last_undegraded":"2024-04-26T00:25:08.964127+0000","last_fullsized":"2024-04-26T00:25:42.858792+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2592,"log_dups_size":0,"ondisk_log_size":2592,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10593222,"num_objects":2588,"num_object_clones":0,"num_object_copies":5176,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"29'2522","reported_seq":2570,"reported_epoch":29,"state":"active+recovering","last_fresh":"2024-04-26T00:25:42.860487+0000","last_change":"2024-04-26T00:25:20.219912+0000","last_active":"2024-04-26T00:25:42.860487+0000","last_peered":"2024-04-26T00:25:42.860487+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:42.860487+0000","last_undegraded":"2024-04-26T00:25:42.860487+0000","last_fullsized":"2024-04-26T00:25:42.860487+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2522,"log_dups_size":0,"ondisk_log_size":2522,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10322403,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2521,"num_write_kb":10081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2521}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973776+0000","last_change":"2024-04-26T00:25:08.973776+0000","last_active":"2024-04-26T00:25:08.973776+0000","last_peered":"2024-04-26T00:25:08.973776+0000","last_clean":"2024-04-26T00:25:08.973776+0000","last_became_active":"2024-04-26T00:25:08.973531+0000","last_became_peered":"2024-04-26T00:25:08.973531+0000","last_unstale":"2024-04-26T00:25:08.973776+0000","last_undegraded":"2024-04-26T00:25:08.973776+0000","last_fullsized":"2024-04-26T00:25:08.973776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:51:51.533562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"29'2510","reported_seq":2530,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.880793+0000","last_change":"2024-04-26T00:25:33.924747+0000","last_active":"2024-04-26T00:25:42.880793+0000","last_peered":"2024-04-26T00:25:42.880793+0000","last_clean":"2024-04-26T00:25:42.880793+0000","last_became_active":"2024-04-26T00:25:08.972082+0000","last_became_peered":"2024-04-26T00:25:08.972082+0000","last_unstale":"2024-04-26T00:25:42.880793+0000","last_undegraded":"2024-04-26T00:25:42.880793+0000","last_fullsized":"2024-04-26T00:25:42.880793+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:44.947059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10253254,"num_objects":2505,"num_object_clones":0,"num_object_copies":5010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2508,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969847+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:08.969847+0000","last_peered":"2024-04-26T00:25:08.969847+0000","last_clean":"2024-04-26T00:25:08.969847+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:08.969847+0000","last_undegraded":"2024-04-26T00:25:08.969847+0000","last_fullsized":"2024-04-26T00:25:08.969847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"29'2617","reported_seq":2658,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.850954+0000","last_change":"2024-04-26T00:25:08.976365+0000","last_active":"2024-04-26T00:25:42.850954+0000","last_peered":"2024-04-26T00:25:42.850954+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:08.976097+0000","last_became_peered":"2024-04-26T00:25:08.976097+0000","last_unstale":"2024-04-26T00:25:42.850954+0000","last_undegraded":"2024-04-26T00:25:08.964101+0000","last_fullsized":"2024-04-26T00:25:42.850954+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2617,"log_dups_size":0,"ondisk_log_size":2617,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10687430,"num_objects":2611,"num_object_clones":0,"num_object_copies":5222,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2615,"num_write_kb":10440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973959+0000","last_change":"2024-04-26T00:25:08.973959+0000","last_active":"2024-04-26T00:25:08.973959+0000","last_peered":"2024-04-26T00:25:08.973959+0000","last_clean":"2024-04-26T00:25:08.973959+0000","last_became_active":"2024-04-26T00:25:08.973682+0000","last_became_peered":"2024-04-26T00:25:08.973682+0000","last_unstale":"2024-04-26T00:25:08.973959+0000","last_undegraded":"2024-04-26T00:25:08.973959+0000","last_fullsized":"2024-04-26T00:25:08.973959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:01.958996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"29'2667","reported_seq":2694,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.886725+0000","last_change":"2024-04-26T00:25:34.995156+0000","last_active":"2024-04-26T00:25:42.886725+0000","last_peered":"2024-04-26T00:25:42.886725+0000","last_clean":"2024-04-26T00:25:42.886725+0000","last_became_active":"2024-04-26T00:25:08.968798+0000","last_became_peered":"2024-04-26T00:25:08.968798+0000","last_unstale":"2024-04-26T00:25:42.886725+0000","last_undegraded":"2024-04-26T00:25:42.886725+0000","last_fullsized":"2024-04-26T00:25:42.886725+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2667,"log_dups_size":0,"ondisk_log_size":2667,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:45.943146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10904518,"num_objects":2664,"num_object_clones":0,"num_object_copies":5328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2665,"num_write_kb":10651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974651+0000","last_change":"2024-04-26T00:25:08.974651+0000","last_active":"2024-04-26T00:25:08.974651+0000","last_peered":"2024-04-26T00:25:08.974651+0000","last_clean":"2024-04-26T00:25:08.974651+0000","last_became_active":"2024-04-26T00:25:08.973555+0000","last_became_peered":"2024-04-26T00:25:08.973555+0000","last_unstale":"2024-04-26T00:25:08.974651+0000","last_undegraded":"2024-04-26T00:25:08.974651+0000","last_fullsized":"2024-04-26T00:25:08.974651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:43.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"29'2565","reported_seq":2591,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.889244+0000","last_change":"2024-04-26T00:25:35.448359+0000","last_active":"2024-04-26T00:25:42.889244+0000","last_peered":"2024-04-26T00:25:42.889244+0000","last_clean":"2024-04-26T00:25:42.889244+0000","last_became_active":"2024-04-26T00:25:08.971629+0000","last_became_peered":"2024-04-26T00:25:08.971629+0000","last_unstale":"2024-04-26T00:25:42.889244+0000","last_undegraded":"2024-04-26T00:25:42.889244+0000","last_fullsized":"2024-04-26T00:25:42.889244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2565,"log_dups_size":0,"ondisk_log_size":2565,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:45.943295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10482630,"num_objects":2561,"num_object_clones":0,"num_object_copies":5122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2563,"num_write_kb":10239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"29'2623","reported_seq":2669,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.836912+0000","last_change":"2024-04-26T00:25:08.976154+0000","last_active":"2024-04-26T00:25:42.836912+0000","last_peered":"2024-04-26T00:25:42.836912+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:08.975847+0000","last_became_peered":"2024-04-26T00:25:08.975847+0000","last_unstale":"2024-04-26T00:25:42.836912+0000","last_undegraded":"2024-04-26T00:25:08.963824+0000","last_fullsized":"2024-04-26T00:25:42.836912+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2623,"log_dups_size":0,"ondisk_log_size":2623,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10736099,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2622}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.970682+0000","last_change":"2024-04-26T00:25:08.970682+0000","last_active":"2024-04-26T00:25:08.970682+0000","last_peered":"2024-04-26T00:25:08.970682+0000","last_clean":"2024-04-26T00:25:08.970682+0000","last_became_active":"2024-04-26T00:25:08.970402+0000","last_became_peered":"2024-04-26T00:25:08.970402+0000","last_unstale":"2024-04-26T00:25:08.970682+0000","last_undegraded":"2024-04-26T00:25:08.970682+0000","last_fullsized":"2024-04-26T00:25:08.970682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:30.870253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"29'2533","reported_seq":2578,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.850760+0000","last_change":"2024-04-26T00:25:08.976265+0000","last_active":"2024-04-26T00:25:42.850760+0000","last_peered":"2024-04-26T00:25:42.850760+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:08.976026+0000","last_became_peered":"2024-04-26T00:25:08.976026+0000","last_unstale":"2024-04-26T00:25:42.850760+0000","last_undegraded":"2024-04-26T00:25:08.964098+0000","last_fullsized":"2024-04-26T00:25:42.850760+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2533,"log_dups_size":0,"ondisk_log_size":2533,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10343849,"num_objects":2528,"num_object_clones":0,"num_object_copies":5056,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2530,"num_write_kb":10104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.975493+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:08.975493+0000","last_peered":"2024-04-26T00:25:08.975493+0000","last_clean":"2024-04-26T00:25:08.975493+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:08.975493+0000","last_undegraded":"2024-04-26T00:25:08.975493+0000","last_fullsized":"2024-04-26T00:25:08.975493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"29'2600","reported_seq":2645,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.812661+0000","last_change":"2024-04-26T00:25:08.976936+0000","last_active":"2024-04-26T00:25:42.812661+0000","last_peered":"2024-04-26T00:25:42.812661+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:08.976619+0000","last_became_peered":"2024-04-26T00:25:08.976619+0000","last_unstale":"2024-04-26T00:25:42.812661+0000","last_undegraded":"2024-04-26T00:25:08.964511+0000","last_fullsized":"2024-04-26T00:25:42.812661+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2600,"log_dups_size":0,"ondisk_log_size":2600,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10634182,"num_objects":2598,"num_object_clones":0,"num_object_copies":5196,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2598,"num_write_kb":10386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974051+0000","last_change":"2024-04-26T00:25:08.974051+0000","last_active":"2024-04-26T00:25:08.974051+0000","last_peered":"2024-04-26T00:25:08.974051+0000","last_clean":"2024-04-26T00:25:08.974051+0000","last_became_active":"2024-04-26T00:25:08.973751+0000","last_became_peered":"2024-04-26T00:25:08.973751+0000","last_unstale":"2024-04-26T00:25:08.974051+0000","last_undegraded":"2024-04-26T00:25:08.974051+0000","last_fullsized":"2024-04-26T00:25:08.974051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:01.153160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974072+0000","last_change":"2024-04-26T00:25:08.974072+0000","last_active":"2024-04-26T00:25:08.974072+0000","last_peered":"2024-04-26T00:25:08.974072+0000","last_clean":"2024-04-26T00:25:08.974072+0000","last_became_active":"2024-04-26T00:25:08.973370+0000","last_became_peered":"2024-04-26T00:25:08.973370+0000","last_unstale":"2024-04-26T00:25:08.974072+0000","last_undegraded":"2024-04-26T00:25:08.974072+0000","last_fullsized":"2024-04-26T00:25:08.974072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:51.793538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"29'2664","reported_seq":2708,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.859391+0000","last_change":"2024-04-26T00:25:08.972679+0000","last_active":"2024-04-26T00:25:42.859391+0000","last_peered":"2024-04-26T00:25:42.859391+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:08.972513+0000","last_became_peered":"2024-04-26T00:25:08.972513+0000","last_unstale":"2024-04-26T00:25:42.859391+0000","last_undegraded":"2024-04-26T00:25:08.963881+0000","last_fullsized":"2024-04-26T00:25:42.859391+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2664,"log_dups_size":0,"ondisk_log_size":2664,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10864524,"num_objects":2656,"num_object_clones":0,"num_object_copies":5312,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2661,"num_write_kb":10615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":170747092,"num_objects":41711,"num_object_clones":0,"num_object_copies":83422,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41738,"num_write_kb":166778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":341741568,"data_stored":341618726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41765,"ondisk_log_size":41765,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274888,"kb_used_data":166904,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94096952,"statfs":{"total":96636764160,"available":96355278848,"internally_reserved":0,"allocated":170909696,"data_stored":170717735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":110554829},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084300,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":275204,"kb_used_data":167164,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94096636,"statfs":{"total":96636764160,"available":96354955264,"internally_reserved":0,"allocated":171175936,"data_stored":171005653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":110620368},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,8,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":170696704,"data_stored":170675026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":170930176,"data_stored":170930176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:45.550 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:46.047 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:46.048 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:46.065 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-04-26T00:25:44.413271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":170747092,"num_objects":41711,"num_object_clones":0,"num_object_copies":83422,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41738,"num_write_kb":166778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41765,"ondisk_log_size":41765,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":550092,"kb_used_data":334068,"kb_used_omap":8,"kb_used_meta":215991,"kb_avail":188193588,"statfs":{"total":193273528320,"available":192710234112,"internally_reserved":0,"allocated":342085632,"data_stored":341723388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":221175197},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,11,8],"upper_bound":32},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":96329728,"num_objects":23518,"num_object_clones":0,"num_object_copies":47036,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":23518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23518,"num_write_kb":94072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002489"},"pg_stats":[{"pgid":"2.f","version":"29'2633","reported_seq":2673,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.850956+0000","last_change":"2024-04-26T00:25:08.976334+0000","last_active":"2024-04-26T00:25:42.850956+0000","last_peered":"2024-04-26T00:25:42.850956+0000","last_clean":"2024-04-26T00:25:42.850956+0000","last_became_active":"2024-04-26T00:25:08.976010+0000","last_became_peered":"2024-04-26T00:25:08.976010+0000","last_unstale":"2024-04-26T00:25:42.850956+0000","last_undegraded":"2024-04-26T00:25:42.850956+0000","last_fullsized":"2024-04-26T00:25:42.850956+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2633,"log_dups_size":0,"ondisk_log_size":2633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:51.146504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10784768,"num_objects":2633,"num_object_clones":0,"num_object_copies":5266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2633,"num_write_kb":10532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"29'2785","reported_seq":2811,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.895772+0000","last_change":"2024-04-26T00:25:31.147930+0000","last_active":"2024-04-26T00:25:42.895772+0000","last_peered":"2024-04-26T00:25:42.895772+0000","last_clean":"2024-04-26T00:25:42.895772+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:42.895772+0000","last_undegraded":"2024-04-26T00:25:42.895772+0000","last_fullsized":"2024-04-26T00:25:42.895772+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2785,"log_dups_size":0,"ondisk_log_size":2785,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.973931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11383750,"num_objects":2781,"num_object_clones":0,"num_object_copies":5562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2783,"num_write_kb":11120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"29'2620","reported_seq":2640,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.847913+0000","last_change":"2024-04-26T00:25:31.575878+0000","last_active":"2024-04-26T00:25:42.847913+0000","last_peered":"2024-04-26T00:25:42.847913+0000","last_clean":"2024-04-26T00:25:42.847913+0000","last_became_active":"2024-04-26T00:25:08.973037+0000","last_became_peered":"2024-04-26T00:25:08.973037+0000","last_unstale":"2024-04-26T00:25:42.847913+0000","last_undegraded":"2024-04-26T00:25:42.847913+0000","last_fullsized":"2024-04-26T00:25:42.847913+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2620,"log_dups_size":0,"ondisk_log_size":2620,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:42.974688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10715619,"num_objects":2617,"num_object_clones":0,"num_object_copies":5234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2619,"num_write_kb":10466,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"29'2573","reported_seq":2592,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.897671+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:42.897671+0000","last_peered":"2024-04-26T00:25:42.897671+0000","last_clean":"2024-04-26T00:25:42.897671+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:42.897671+0000","last_undegraded":"2024-04-26T00:25:42.897671+0000","last_fullsized":"2024-04-26T00:25:42.897671+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2573,"log_dups_size":0,"ondisk_log_size":2573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10539008,"num_objects":2573,"num_object_clones":0,"num_object_copies":5146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2573,"num_write_kb":10292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"29'2669","reported_seq":2714,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.786855+0000","last_change":"2024-04-26T00:25:08.974060+0000","last_active":"2024-04-26T00:25:42.786855+0000","last_peered":"2024-04-26T00:25:42.786855+0000","last_clean":"2024-04-26T00:24:44.361957+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:42.786855+0000","last_undegraded":"2024-04-26T00:25:08.963569+0000","last_fullsized":"2024-04-26T00:25:42.786855+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2669,"log_dups_size":0,"ondisk_log_size":2669,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10904518,"num_objects":2664,"num_object_clones":0,"num_object_copies":5328,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2667,"num_write_kb":10652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2664}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"29'2592","reported_seq":2638,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.862424+0000","last_change":"2024-04-26T00:25:08.972675+0000","last_active":"2024-04-26T00:25:42.862424+0000","last_peered":"2024-04-26T00:25:42.862424+0000","last_clean":"2024-04-26T00:24:45.146001+0000","last_became_active":"2024-04-26T00:25:08.972454+0000","last_became_peered":"2024-04-26T00:25:08.972454+0000","last_unstale":"2024-04-26T00:25:42.862424+0000","last_undegraded":"2024-04-26T00:25:08.963672+0000","last_fullsized":"2024-04-26T00:25:42.862424+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2592,"log_dups_size":0,"ondisk_log_size":2592,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10597318,"num_objects":2589,"num_object_clones":0,"num_object_copies":5178,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2589,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10351,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2589}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"29'2592","reported_seq":2637,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.858792+0000","last_change":"2024-04-26T00:25:08.970817+0000","last_active":"2024-04-26T00:25:42.858792+0000","last_peered":"2024-04-26T00:25:42.858792+0000","last_clean":"2024-04-26T00:24:44.591091+0000","last_became_active":"2024-04-26T00:25:08.970587+0000","last_became_peered":"2024-04-26T00:25:08.970587+0000","last_unstale":"2024-04-26T00:25:42.858792+0000","last_undegraded":"2024-04-26T00:25:08.964127+0000","last_fullsized":"2024-04-26T00:25:42.858792+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2592,"log_dups_size":0,"ondisk_log_size":2592,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10593222,"num_objects":2588,"num_object_clones":0,"num_object_copies":5176,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2588}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"29'2522","reported_seq":2570,"reported_epoch":29,"state":"active+recovering","last_fresh":"2024-04-26T00:25:42.860487+0000","last_change":"2024-04-26T00:25:20.219912+0000","last_active":"2024-04-26T00:25:42.860487+0000","last_peered":"2024-04-26T00:25:42.860487+0000","last_clean":"2024-04-26T00:24:34.529933+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:42.860487+0000","last_undegraded":"2024-04-26T00:25:42.860487+0000","last_fullsized":"2024-04-26T00:25:42.860487+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2522,"log_dups_size":0,"ondisk_log_size":2522,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10322403,"num_objects":2521,"num_object_clones":0,"num_object_copies":5042,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2521,"num_write_kb":10081,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2521}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973776+0000","last_change":"2024-04-26T00:25:08.973776+0000","last_active":"2024-04-26T00:25:08.973776+0000","last_peered":"2024-04-26T00:25:08.973776+0000","last_clean":"2024-04-26T00:25:08.973776+0000","last_became_active":"2024-04-26T00:25:08.973531+0000","last_became_peered":"2024-04-26T00:25:08.973531+0000","last_unstale":"2024-04-26T00:25:08.973776+0000","last_undegraded":"2024-04-26T00:25:08.973776+0000","last_fullsized":"2024-04-26T00:25:08.973776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:51:51.533562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"29'2510","reported_seq":2530,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.880793+0000","last_change":"2024-04-26T00:25:33.924747+0000","last_active":"2024-04-26T00:25:42.880793+0000","last_peered":"2024-04-26T00:25:42.880793+0000","last_clean":"2024-04-26T00:25:42.880793+0000","last_became_active":"2024-04-26T00:25:08.972082+0000","last_became_peered":"2024-04-26T00:25:08.972082+0000","last_unstale":"2024-04-26T00:25:42.880793+0000","last_undegraded":"2024-04-26T00:25:42.880793+0000","last_fullsized":"2024-04-26T00:25:42.880793+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2510,"log_dups_size":0,"ondisk_log_size":2510,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:44.947059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10253254,"num_objects":2505,"num_object_clones":0,"num_object_copies":5010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2508,"num_write_kb":10016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969847+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:08.969847+0000","last_peered":"2024-04-26T00:25:08.969847+0000","last_clean":"2024-04-26T00:25:08.969847+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:08.969847+0000","last_undegraded":"2024-04-26T00:25:08.969847+0000","last_fullsized":"2024-04-26T00:25:08.969847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"29'2617","reported_seq":2658,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.850954+0000","last_change":"2024-04-26T00:25:08.976365+0000","last_active":"2024-04-26T00:25:42.850954+0000","last_peered":"2024-04-26T00:25:42.850954+0000","last_clean":"2024-04-26T00:24:45.192411+0000","last_became_active":"2024-04-26T00:25:08.976097+0000","last_became_peered":"2024-04-26T00:25:08.976097+0000","last_unstale":"2024-04-26T00:25:42.850954+0000","last_undegraded":"2024-04-26T00:25:08.964101+0000","last_fullsized":"2024-04-26T00:25:42.850954+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2617,"log_dups_size":0,"ondisk_log_size":2617,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10687430,"num_objects":2611,"num_object_clones":0,"num_object_copies":5222,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2615,"num_write_kb":10440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2611}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973959+0000","last_change":"2024-04-26T00:25:08.973959+0000","last_active":"2024-04-26T00:25:08.973959+0000","last_peered":"2024-04-26T00:25:08.973959+0000","last_clean":"2024-04-26T00:25:08.973959+0000","last_became_active":"2024-04-26T00:25:08.973682+0000","last_became_peered":"2024-04-26T00:25:08.973682+0000","last_unstale":"2024-04-26T00:25:08.973959+0000","last_undegraded":"2024-04-26T00:25:08.973959+0000","last_fullsized":"2024-04-26T00:25:08.973959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:01.958996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"29'2667","reported_seq":2694,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.886725+0000","last_change":"2024-04-26T00:25:34.995156+0000","last_active":"2024-04-26T00:25:42.886725+0000","last_peered":"2024-04-26T00:25:42.886725+0000","last_clean":"2024-04-26T00:25:42.886725+0000","last_became_active":"2024-04-26T00:25:08.968798+0000","last_became_peered":"2024-04-26T00:25:08.968798+0000","last_unstale":"2024-04-26T00:25:42.886725+0000","last_undegraded":"2024-04-26T00:25:42.886725+0000","last_fullsized":"2024-04-26T00:25:42.886725+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2667,"log_dups_size":0,"ondisk_log_size":2667,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:45.943146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10904518,"num_objects":2664,"num_object_clones":0,"num_object_copies":5328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2665,"num_write_kb":10651,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974651+0000","last_change":"2024-04-26T00:25:08.974651+0000","last_active":"2024-04-26T00:25:08.974651+0000","last_peered":"2024-04-26T00:25:08.974651+0000","last_clean":"2024-04-26T00:25:08.974651+0000","last_became_active":"2024-04-26T00:25:08.973555+0000","last_became_peered":"2024-04-26T00:25:08.973555+0000","last_unstale":"2024-04-26T00:25:08.974651+0000","last_undegraded":"2024-04-26T00:25:08.974651+0000","last_fullsized":"2024-04-26T00:25:08.974651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:43.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"29'2565","reported_seq":2591,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:42.889244+0000","last_change":"2024-04-26T00:25:35.448359+0000","last_active":"2024-04-26T00:25:42.889244+0000","last_peered":"2024-04-26T00:25:42.889244+0000","last_clean":"2024-04-26T00:25:42.889244+0000","last_became_active":"2024-04-26T00:25:08.971629+0000","last_became_peered":"2024-04-26T00:25:08.971629+0000","last_unstale":"2024-04-26T00:25:42.889244+0000","last_undegraded":"2024-04-26T00:25:42.889244+0000","last_fullsized":"2024-04-26T00:25:42.889244+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2565,"log_dups_size":0,"ondisk_log_size":2565,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:25:45.943295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10482630,"num_objects":2561,"num_object_clones":0,"num_object_copies":5122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2563,"num_write_kb":10239,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"29'2623","reported_seq":2669,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.836912+0000","last_change":"2024-04-26T00:25:08.976154+0000","last_active":"2024-04-26T00:25:42.836912+0000","last_peered":"2024-04-26T00:25:42.836912+0000","last_clean":"2024-04-26T00:24:37.739490+0000","last_became_active":"2024-04-26T00:25:08.975847+0000","last_became_peered":"2024-04-26T00:25:08.975847+0000","last_unstale":"2024-04-26T00:25:42.836912+0000","last_undegraded":"2024-04-26T00:25:08.963824+0000","last_fullsized":"2024-04-26T00:25:42.836912+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2623,"log_dups_size":0,"ondisk_log_size":2623,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10736099,"num_objects":2622,"num_object_clones":0,"num_object_copies":5244,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2622,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2622,"num_write_kb":10485,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2622}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.970682+0000","last_change":"2024-04-26T00:25:08.970682+0000","last_active":"2024-04-26T00:25:08.970682+0000","last_peered":"2024-04-26T00:25:08.970682+0000","last_clean":"2024-04-26T00:25:08.970682+0000","last_became_active":"2024-04-26T00:25:08.970402+0000","last_became_peered":"2024-04-26T00:25:08.970402+0000","last_unstale":"2024-04-26T00:25:08.970682+0000","last_undegraded":"2024-04-26T00:25:08.970682+0000","last_fullsized":"2024-04-26T00:25:08.970682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:30.870253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"29'2533","reported_seq":2578,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.850760+0000","last_change":"2024-04-26T00:25:08.976265+0000","last_active":"2024-04-26T00:25:42.850760+0000","last_peered":"2024-04-26T00:25:42.850760+0000","last_clean":"2024-04-26T00:24:45.099969+0000","last_became_active":"2024-04-26T00:25:08.976026+0000","last_became_peered":"2024-04-26T00:25:08.976026+0000","last_unstale":"2024-04-26T00:25:42.850760+0000","last_undegraded":"2024-04-26T00:25:08.964098+0000","last_fullsized":"2024-04-26T00:25:42.850760+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2533,"log_dups_size":0,"ondisk_log_size":2533,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10343849,"num_objects":2528,"num_object_clones":0,"num_object_copies":5056,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2530,"num_write_kb":10104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2528}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.975493+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:08.975493+0000","last_peered":"2024-04-26T00:25:08.975493+0000","last_clean":"2024-04-26T00:25:08.975493+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:08.975493+0000","last_undegraded":"2024-04-26T00:25:08.975493+0000","last_fullsized":"2024-04-26T00:25:08.975493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"29'2600","reported_seq":2645,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.812661+0000","last_change":"2024-04-26T00:25:08.976936+0000","last_active":"2024-04-26T00:25:42.812661+0000","last_peered":"2024-04-26T00:25:42.812661+0000","last_clean":"2024-04-26T00:24:37.739271+0000","last_became_active":"2024-04-26T00:25:08.976619+0000","last_became_peered":"2024-04-26T00:25:08.976619+0000","last_unstale":"2024-04-26T00:25:42.812661+0000","last_undegraded":"2024-04-26T00:25:08.964511+0000","last_fullsized":"2024-04-26T00:25:42.812661+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2600,"log_dups_size":0,"ondisk_log_size":2600,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10634182,"num_objects":2598,"num_object_clones":0,"num_object_copies":5196,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2598,"num_write_kb":10386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2598}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974051+0000","last_change":"2024-04-26T00:25:08.974051+0000","last_active":"2024-04-26T00:25:08.974051+0000","last_peered":"2024-04-26T00:25:08.974051+0000","last_clean":"2024-04-26T00:25:08.974051+0000","last_became_active":"2024-04-26T00:25:08.973751+0000","last_became_peered":"2024-04-26T00:25:08.973751+0000","last_unstale":"2024-04-26T00:25:08.974051+0000","last_undegraded":"2024-04-26T00:25:08.974051+0000","last_fullsized":"2024-04-26T00:25:08.974051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:01.153160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974072+0000","last_change":"2024-04-26T00:25:08.974072+0000","last_active":"2024-04-26T00:25:08.974072+0000","last_peered":"2024-04-26T00:25:08.974072+0000","last_clean":"2024-04-26T00:25:08.974072+0000","last_became_active":"2024-04-26T00:25:08.973370+0000","last_became_peered":"2024-04-26T00:25:08.973370+0000","last_unstale":"2024-04-26T00:25:08.974072+0000","last_undegraded":"2024-04-26T00:25:08.974072+0000","last_fullsized":"2024-04-26T00:25:08.974072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:51.793538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"29'2664","reported_seq":2708,"reported_epoch":29,"state":"active+recovery_wait+degraded","last_fresh":"2024-04-26T00:25:42.859391+0000","last_change":"2024-04-26T00:25:08.972679+0000","last_active":"2024-04-26T00:25:42.859391+0000","last_peered":"2024-04-26T00:25:42.859391+0000","last_clean":"2024-04-26T00:24:44.871080+0000","last_became_active":"2024-04-26T00:25:08.972513+0000","last_became_peered":"2024-04-26T00:25:08.972513+0000","last_unstale":"2024-04-26T00:25:42.859391+0000","last_undegraded":"2024-04-26T00:25:08.963881+0000","last_fullsized":"2024-04-26T00:25:42.859391+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2664,"log_dups_size":0,"ondisk_log_size":2664,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10864524,"num_objects":2656,"num_object_clones":0,"num_object_copies":5312,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2661,"num_write_kb":10615,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2655}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":170747092,"num_objects":41711,"num_object_clones":0,"num_object_copies":83422,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":35,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41738,"num_write_kb":166778,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":341741568,"data_stored":341618726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41765,"ondisk_log_size":41765,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280527,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":274888,"kb_used_data":166904,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94096952,"statfs":{"total":96636764160,"available":96355278848,"internally_reserved":0,"allocated":170909696,"data_stored":170717735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":110554829},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084300,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":275204,"kb_used_data":167164,"kb_used_omap":4,"kb_used_meta":108027,"kb_avail":94096636,"statfs":{"total":96636764160,"available":96354955264,"internally_reserved":0,"allocated":171175936,"data_stored":171005653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":110620368},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,8,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":170696704,"data_stored":170675026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":170930176,"data_stored":170930176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:46.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 26 20 45864 45844 6.88705 4.42188 0.0108871 0.0113376 2024-04-26T00:25:47.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 27 20 46719 46699 6.75566 3.33984 0.023104 0.0115575 2024-04-26T00:25:48.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 28 20 47877 47857 6.67592 4.52344 0.0351727 0.0116947 2024-04-26T00:25:49.066 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:25:49.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 29 20 49021 49001 6.5998 4.46875 0.0157956 0.0118307 2024-04-26T00:25:49.574 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:49.575 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:25:49.597 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":70,"stamp":"2024-04-26T00:25:48.414010+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":194706992,"num_objects":47550,"num_object_clones":0,"num_object_copies":95100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30321,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47616,"ondisk_log_size":47616,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":604724,"kb_used_data":376228,"kb_used_omap":8,"kb_used_meta":228471,"kb_avail":188138956,"statfs":{"total":193273528320,"available":192654290944,"internally_reserved":0,"allocated":385257472,"data_stored":384909680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8803,"internal_metadata":233954717},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,2,8,9,9,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":80968028,"num_objects":19757,"num_object_clones":0,"num_object_copies":39514,"num_objects_missing_on_primary":-24,"num_objects_missing":-24,"num_objects_degraded":-47,"num_objects_misplaced":0,"num_objects_unfound":-20,"num_objects_dirty":19757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2501,"num_write_kb":10085,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002285"},"pg_stats":[{"pgid":"2.f","version":"29'3002","reported_seq":3042,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.905374+0000","last_change":"2024-04-26T00:25:08.976334+0000","last_active":"2024-04-26T00:25:47.905374+0000","last_peered":"2024-04-26T00:25:47.905374+0000","last_clean":"2024-04-26T00:25:47.905374+0000","last_became_active":"2024-04-26T00:25:08.976010+0000","last_became_peered":"2024-04-26T00:25:08.976010+0000","last_unstale":"2024-04-26T00:25:47.905374+0000","last_undegraded":"2024-04-26T00:25:47.905374+0000","last_fullsized":"2024-04-26T00:25:47.905374+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3002,"log_dups_size":0,"ondisk_log_size":3002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:51.146504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12296192,"num_objects":3002,"num_object_clones":0,"num_object_copies":6004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3002,"num_write_kb":12008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"29'3155","reported_seq":3218,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.960701+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:25:47.960701+0000","last_peered":"2024-04-26T00:25:47.960701+0000","last_clean":"2024-04-26T00:25:47.960701+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:47.960701+0000","last_undegraded":"2024-04-26T00:25:47.960701+0000","last_fullsized":"2024-04-26T00:25:47.960701+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":3155,"log_dups_size":0,"ondisk_log_size":3155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12898304,"num_objects":3149,"num_object_clones":0,"num_object_copies":6298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":260,"num_write_kb":1040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"29'2987","reported_seq":3043,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.978421+0000","last_change":"2024-04-26T00:25:45.686814+0000","last_active":"2024-04-26T00:25:47.978421+0000","last_peered":"2024-04-26T00:25:47.978421+0000","last_clean":"2024-04-26T00:25:47.978421+0000","last_became_active":"2024-04-26T00:25:08.973037+0000","last_became_peered":"2024-04-26T00:25:08.973037+0000","last_unstale":"2024-04-26T00:25:47.978421+0000","last_undegraded":"2024-04-26T00:25:47.978421+0000","last_fullsized":"2024-04-26T00:25:47.978421+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":2987,"log_dups_size":0,"ondisk_log_size":2987,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:19:13.950347+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12218368,"num_objects":2983,"num_object_clones":0,"num_object_copies":5966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"29'2914","reported_seq":2933,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.960777+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:47.960777+0000","last_peered":"2024-04-26T00:25:47.960777+0000","last_clean":"2024-04-26T00:25:47.960777+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:47.960777+0000","last_undegraded":"2024-04-26T00:25:47.960777+0000","last_fullsized":"2024-04-26T00:25:47.960777+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2914,"log_dups_size":0,"ondisk_log_size":2914,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11935744,"num_objects":2914,"num_object_clones":0,"num_object_copies":5828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2914,"num_write_kb":11656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"29'3046","reported_seq":3100,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.900735+0000","last_change":"2024-04-26T00:25:43.319001+0000","last_active":"2024-04-26T00:25:47.900735+0000","last_peered":"2024-04-26T00:25:47.900735+0000","last_clean":"2024-04-26T00:25:47.900735+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:47.900735+0000","last_undegraded":"2024-04-26T00:25:47.900735+0000","last_fullsized":"2024-04-26T00:25:47.900735+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3046,"log_dups_size":0,"ondisk_log_size":3046,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12448710,"num_objects":3041,"num_object_clones":0,"num_object_copies":6082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3044,"num_write_kb":12160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"29'2928","reported_seq":3020,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.915949+0000","last_change":"2024-04-26T00:25:46.820032+0000","last_active":"2024-04-26T00:25:47.915949+0000","last_peered":"2024-04-26T00:25:47.915949+0000","last_clean":"2024-04-26T00:25:47.915949+0000","last_became_active":"2024-04-26T00:25:08.972454+0000","last_became_peered":"2024-04-26T00:25:08.972454+0000","last_unstale":"2024-04-26T00:25:47.915949+0000","last_undegraded":"2024-04-26T00:25:47.915949+0000","last_fullsized":"2024-04-26T00:25:47.915949+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":2928,"log_dups_size":0,"ondisk_log_size":2928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:35:53.084144+0000","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11972608,"num_objects":2923,"num_object_clones":0,"num_object_copies":5846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":103,"num_write_kb":412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"29'2955","reported_seq":3048,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.898519+0000","last_change":"2024-04-26T00:25:45.706575+0000","last_active":"2024-04-26T00:25:47.898519+0000","last_peered":"2024-04-26T00:25:47.898519+0000","last_clean":"2024-04-26T00:25:47.898519+0000","last_became_active":"2024-04-26T00:25:08.970587+0000","last_became_peered":"2024-04-26T00:25:08.970587+0000","last_unstale":"2024-04-26T00:25:47.898519+0000","last_undegraded":"2024-04-26T00:25:47.898519+0000","last_fullsized":"2024-04-26T00:25:47.898519+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":2955,"log_dups_size":0,"ondisk_log_size":2955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:38:47.537366+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12079104,"num_objects":2949,"num_object_clones":0,"num_object_copies":5898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":157,"num_write_kb":628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"29'2886","reported_seq":3034,"reported_epoch":29,"state":"active+clean+scrubbing+deep","last_fresh":"2024-04-26T00:25:47.888794+0000","last_change":"2024-04-26T00:25:43.849898+0000","last_active":"2024-04-26T00:25:47.888794+0000","last_peered":"2024-04-26T00:25:47.888794+0000","last_clean":"2024-04-26T00:25:47.888794+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:47.888794+0000","last_undegraded":"2024-04-26T00:25:47.888794+0000","last_fullsized":"2024-04-26T00:25:47.888794+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":2236,"log_size":2886,"log_dups_size":0,"ondisk_log_size":2886,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 4s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11813347,"num_objects":2885,"num_object_clones":0,"num_object_copies":5770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2885,"num_write_kb":11537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973776+0000","last_change":"2024-04-26T00:25:08.973776+0000","last_active":"2024-04-26T00:25:08.973776+0000","last_peered":"2024-04-26T00:25:08.973776+0000","last_clean":"2024-04-26T00:25:08.973776+0000","last_became_active":"2024-04-26T00:25:08.973531+0000","last_became_peered":"2024-04-26T00:25:08.973531+0000","last_unstale":"2024-04-26T00:25:08.973776+0000","last_undegraded":"2024-04-26T00:25:08.973776+0000","last_fullsized":"2024-04-26T00:25:08.973776+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:51:51.533562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"29'2859","reported_seq":2927,"reported_epoch":29,"state":"active+clean+scrubbing+deep","last_fresh":"2024-04-26T00:25:47.978871+0000","last_change":"2024-04-26T00:25:45.935034+0000","last_active":"2024-04-26T00:25:47.978871+0000","last_peered":"2024-04-26T00:25:47.978871+0000","last_clean":"2024-04-26T00:25:47.978871+0000","last_became_active":"2024-04-26T00:25:08.972082+0000","last_became_peered":"2024-04-26T00:25:08.972082+0000","last_unstale":"2024-04-26T00:25:47.978871+0000","last_undegraded":"2024-04-26T00:25:47.978871+0000","last_fullsized":"2024-04-26T00:25:47.978871+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1174,"log_size":2859,"log_dups_size":0,"ondisk_log_size":2859,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11682758,"num_objects":2854,"num_object_clones":0,"num_object_copies":5708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2857,"num_write_kb":11412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.969847+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:08.969847+0000","last_peered":"2024-04-26T00:25:08.969847+0000","last_clean":"2024-04-26T00:25:08.969847+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:08.969847+0000","last_undegraded":"2024-04-26T00:25:08.969847+0000","last_fullsized":"2024-04-26T00:25:08.969847+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"29'3001","reported_seq":3052,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.885255+0000","last_change":"2024-04-26T00:25:43.498574+0000","last_active":"2024-04-26T00:25:47.885255+0000","last_peered":"2024-04-26T00:25:47.885255+0000","last_clean":"2024-04-26T00:25:47.885255+0000","last_became_active":"2024-04-26T00:25:08.976097+0000","last_became_peered":"2024-04-26T00:25:08.976097+0000","last_unstale":"2024-04-26T00:25:47.885255+0000","last_undegraded":"2024-04-26T00:25:47.885255+0000","last_fullsized":"2024-04-26T00:25:47.885255+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3001,"log_dups_size":0,"ondisk_log_size":3001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12260294,"num_objects":2995,"num_object_clones":0,"num_object_copies":5990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2999,"num_write_kb":11976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.973959+0000","last_change":"2024-04-26T00:25:08.973959+0000","last_active":"2024-04-26T00:25:08.973959+0000","last_peered":"2024-04-26T00:25:08.973959+0000","last_clean":"2024-04-26T00:25:08.973959+0000","last_became_active":"2024-04-26T00:25:08.973682+0000","last_became_peered":"2024-04-26T00:25:08.973682+0000","last_unstale":"2024-04-26T00:25:08.973959+0000","last_undegraded":"2024-04-26T00:25:08.973959+0000","last_fullsized":"2024-04-26T00:25:08.973959+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:01.958996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"29'3043","reported_seq":3108,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.971698+0000","last_change":"2024-04-26T00:25:47.737008+0000","last_active":"2024-04-26T00:25:47.971698+0000","last_peered":"2024-04-26T00:25:47.971698+0000","last_clean":"2024-04-26T00:25:47.971698+0000","last_became_active":"2024-04-26T00:25:08.968798+0000","last_became_peered":"2024-04-26T00:25:08.968798+0000","last_unstale":"2024-04-26T00:25:47.971698+0000","last_undegraded":"2024-04-26T00:25:47.971698+0000","last_fullsized":"2024-04-26T00:25:47.971698+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":3043,"log_dups_size":0,"ondisk_log_size":3043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:47:19.161553+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12443648,"num_objects":3038,"num_object_clones":0,"num_object_copies":6076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":43,"num_write_kb":172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974651+0000","last_change":"2024-04-26T00:25:08.974651+0000","last_active":"2024-04-26T00:25:08.974651+0000","last_peered":"2024-04-26T00:25:08.974651+0000","last_clean":"2024-04-26T00:25:08.974651+0000","last_became_active":"2024-04-26T00:25:08.973555+0000","last_became_peered":"2024-04-26T00:25:08.973555+0000","last_unstale":"2024-04-26T00:25:08.974651+0000","last_undegraded":"2024-04-26T00:25:08.974651+0000","last_fullsized":"2024-04-26T00:25:08.974651+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:48:43.232898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"29'2932","reported_seq":2959,"reported_epoch":29,"state":"active+clean+scrubbing","last_fresh":"2024-04-26T00:25:47.982186+0000","last_change":"2024-04-26T00:25:47.984376+0000","last_active":"2024-04-26T00:25:47.982186+0000","last_peered":"2024-04-26T00:25:47.982186+0000","last_clean":"2024-04-26T00:25:47.982186+0000","last_became_active":"2024-04-26T00:25:08.971629+0000","last_became_peered":"2024-04-26T00:25:08.971629+0000","last_unstale":"2024-04-26T00:25:47.982186+0000","last_undegraded":"2024-04-26T00:25:47.982186+0000","last_fullsized":"2024-04-26T00:25:47.982186+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2932,"log_dups_size":0,"ondisk_log_size":2932,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11985862,"num_objects":2928,"num_object_clones":0,"num_object_copies":5856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2930,"num_write_kb":11707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"29'2989","reported_seq":3041,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.914687+0000","last_change":"2024-04-26T00:25:43.357582+0000","last_active":"2024-04-26T00:25:47.914687+0000","last_peered":"2024-04-26T00:25:47.914687+0000","last_clean":"2024-04-26T00:25:47.914687+0000","last_became_active":"2024-04-26T00:25:08.975847+0000","last_became_peered":"2024-04-26T00:25:08.975847+0000","last_unstale":"2024-04-26T00:25:47.914687+0000","last_undegraded":"2024-04-26T00:25:47.914687+0000","last_fullsized":"2024-04-26T00:25:47.914687+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2989,"log_dups_size":0,"ondisk_log_size":2989,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12235235,"num_objects":2988,"num_object_clones":0,"num_object_copies":5976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2988,"num_write_kb":11949,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.970682+0000","last_change":"2024-04-26T00:25:08.970682+0000","last_active":"2024-04-26T00:25:08.970682+0000","last_peered":"2024-04-26T00:25:08.970682+0000","last_clean":"2024-04-26T00:25:08.970682+0000","last_became_active":"2024-04-26T00:25:08.970402+0000","last_became_peered":"2024-04-26T00:25:08.970402+0000","last_unstale":"2024-04-26T00:25:08.970682+0000","last_undegraded":"2024-04-26T00:25:08.970682+0000","last_fullsized":"2024-04-26T00:25:08.970682+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:30.870253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"29'2926","reported_seq":2984,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.895832+0000","last_change":"2024-04-26T00:25:43.437645+0000","last_active":"2024-04-26T00:25:47.895832+0000","last_peered":"2024-04-26T00:25:47.895832+0000","last_clean":"2024-04-26T00:25:47.895832+0000","last_became_active":"2024-04-26T00:25:08.976026+0000","last_became_peered":"2024-04-26T00:25:08.976026+0000","last_unstale":"2024-04-26T00:25:47.895832+0000","last_undegraded":"2024-04-26T00:25:47.895832+0000","last_fullsized":"2024-04-26T00:25:47.895832+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2926,"log_dups_size":0,"ondisk_log_size":2926,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11953577,"num_objects":2921,"num_object_clones":0,"num_object_copies":5842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2923,"num_write_kb":11676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.975493+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:08.975493+0000","last_peered":"2024-04-26T00:25:08.975493+0000","last_clean":"2024-04-26T00:25:08.975493+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:08.975493+0000","last_undegraded":"2024-04-26T00:25:08.975493+0000","last_fullsized":"2024-04-26T00:25:08.975493+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"29'2989","reported_seq":3044,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.909206+0000","last_change":"2024-04-26T00:25:43.545806+0000","last_active":"2024-04-26T00:25:47.909206+0000","last_peered":"2024-04-26T00:25:47.909206+0000","last_clean":"2024-04-26T00:25:47.909206+0000","last_became_active":"2024-04-26T00:25:08.976619+0000","last_became_peered":"2024-04-26T00:25:08.976619+0000","last_unstale":"2024-04-26T00:25:47.909206+0000","last_undegraded":"2024-04-26T00:25:47.909206+0000","last_fullsized":"2024-04-26T00:25:47.909206+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":2989,"log_dups_size":0,"ondisk_log_size":2989,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12227526,"num_objects":2987,"num_object_clones":0,"num_object_copies":5974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2987,"num_write_kb":11942,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":54,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974051+0000","last_change":"2024-04-26T00:25:08.974051+0000","last_active":"2024-04-26T00:25:08.974051+0000","last_peered":"2024-04-26T00:25:08.974051+0000","last_clean":"2024-04-26T00:25:08.974051+0000","last_became_active":"2024-04-26T00:25:08.973751+0000","last_became_peered":"2024-04-26T00:25:08.973751+0000","last_unstale":"2024-04-26T00:25:08.974051+0000","last_undegraded":"2024-04-26T00:25:08.974051+0000","last_fullsized":"2024-04-26T00:25:08.974051+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:09:01.153160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":13,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:08.974072+0000","last_change":"2024-04-26T00:25:08.974072+0000","last_active":"2024-04-26T00:25:08.974072+0000","last_peered":"2024-04-26T00:25:08.974072+0000","last_clean":"2024-04-26T00:25:08.974072+0000","last_became_active":"2024-04-26T00:25:08.973370+0000","last_became_peered":"2024-04-26T00:25:08.973370+0000","last_unstale":"2024-04-26T00:25:08.974072+0000","last_undegraded":"2024-04-26T00:25:08.974072+0000","last_fullsized":"2024-04-26T00:25:08.974072+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:09:51.793538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"29'3004","reported_seq":3101,"reported_epoch":29,"state":"active+clean","last_fresh":"2024-04-26T00:25:47.895693+0000","last_change":"2024-04-26T00:25:47.549898+0000","last_active":"2024-04-26T00:25:47.895693+0000","last_peered":"2024-04-26T00:25:47.895693+0000","last_clean":"2024-04-26T00:25:47.895693+0000","last_became_active":"2024-04-26T00:25:08.972513+0000","last_became_peered":"2024-04-26T00:25:08.972513+0000","last_unstale":"2024-04-26T00:25:47.895693+0000","last_undegraded":"2024-04-26T00:25:47.895693+0000","last_fullsized":"2024-04-26T00:25:47.895693+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":3004,"log_dups_size":0,"ondisk_log_size":3004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:02:19.778790+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12255715,"num_objects":2993,"num_object_clones":0,"num_object_copies":5986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":54,"num_write_kb":216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":194706992,"num_objects":47550,"num_object_clones":0,"num_object_copies":95100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30321,"num_write_kb":121191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":389685248,"data_stored":389576858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47616,"ondisk_log_size":47616,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280528,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301916,"kb_used_data":185620,"kb_used_omap":4,"kb_used_meta":116283,"kb_avail":94069924,"statfs":{"total":96636764160,"available":96327602176,"internally_reserved":0,"allocated":190074880,"data_stored":189900984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":119074509},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,3,5,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084301,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302808,"kb_used_data":190608,"kb_used_omap":4,"kb_used_meta":112187,"kb_avail":94069032,"statfs":{"total":96636764160,"available":96326688768,"internally_reserved":0,"allocated":195182592,"data_stored":195008696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":114880208},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,4,6,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":194588672,"data_stored":194585059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":194981888,"data_stored":194978275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:25:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-04-26T00:25:49.599 INFO:tasks.util.rados:rados -p unfounddel_pool get new_1 - 2024-04-26T00:25:49.599 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_1 - 2024-04-26T00:25:49.678 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_1: (2) No such file or directory 2024-04-26T00:25:49.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:49.684 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_1 - 2024-04-26T00:25:49.684 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_1 - 2024-04-26T00:25:49.804 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_1: (2) No such file or directory 2024-04-26T00:25:49.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:49.808 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_1 - 2024-04-26T00:25:49.808 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_1 - 2024-04-26T00:25:49.891 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_1: (2) No such file or directory 2024-04-26T00:25:49.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:49.896 INFO:tasks.util.rados:rados -p unfounddel_pool get new_2 - 2024-04-26T00:25:49.896 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_2 - 2024-04-26T00:25:49.982 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_2: (2) No such file or directory 2024-04-26T00:25:49.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:49.985 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_2 - 2024-04-26T00:25:49.985 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_2 - 2024-04-26T00:25:50.059 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_2: (2) No such file or directory 2024-04-26T00:25:50.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:50.064 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_2 - 2024-04-26T00:25:50.065 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_2 - 2024-04-26T00:25:50.143 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_2: (2) No such file or directory 2024-04-26T00:25:50.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:50.153 INFO:tasks.util.rados:rados -p unfounddel_pool get new_3 - 2024-04-26T00:25:50.153 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_3 - 2024-04-26T00:25:50.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 30 20 50089 50069 6.51886 4.17188 0.0157678 0.0119755 2024-04-26T00:25:50.249 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_3: (2) No such file or directory 2024-04-26T00:25:50.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:50.255 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_3 - 2024-04-26T00:25:50.255 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_3 - 2024-04-26T00:25:50.374 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_3: (2) No such file or directory 2024-04-26T00:25:50.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:50.383 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_3 - 2024-04-26T00:25:50.383 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_3 - 2024-04-26T00:25:50.469 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_3: (2) No such file or directory 2024-04-26T00:25:50.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:50.474 INFO:tasks.util.rados:rados -p unfounddel_pool get new_4 - 2024-04-26T00:25:50.475 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_4 - 2024-04-26T00:25:50.559 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_4: (2) No such file or directory 2024-04-26T00:25:50.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:50.570 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_4 - 2024-04-26T00:25:50.570 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_4 - 2024-04-26T00:25:50.650 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_4: (2) No such file or directory 2024-04-26T00:25:50.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:50.656 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_4 - 2024-04-26T00:25:50.656 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_4 - 2024-04-26T00:25:50.733 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_4: (2) No such file or directory 2024-04-26T00:25:50.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:50.739 INFO:tasks.util.rados:rados -p unfounddel_pool get new_5 - 2024-04-26T00:25:50.739 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_5 - 2024-04-26T00:25:51.065 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_5: (2) No such file or directory 2024-04-26T00:25:51.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.069 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_5 - 2024-04-26T00:25:51.069 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_5 - 2024-04-26T00:25:51.156 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_5: (2) No such file or directory 2024-04-26T00:25:51.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.163 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_5 - 2024-04-26T00:25:51.164 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_5 - 2024-04-26T00:25:51.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 31 20 50920 50900 6.41328 3.24609 0.0622948 0.0121717 2024-04-26T00:25:51.257 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_5: (2) No such file or directory 2024-04-26T00:25:51.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.262 INFO:tasks.util.rados:rados -p unfounddel_pool get new_6 - 2024-04-26T00:25:51.263 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_6 - 2024-04-26T00:25:51.380 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_6: (2) No such file or directory 2024-04-26T00:25:51.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.389 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_6 - 2024-04-26T00:25:51.389 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_6 - 2024-04-26T00:25:51.483 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_6: (2) No such file or directory 2024-04-26T00:25:51.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.490 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_6 - 2024-04-26T00:25:51.490 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_6 - 2024-04-26T00:25:51.574 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_6: (2) No such file or directory 2024-04-26T00:25:51.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.580 INFO:tasks.util.rados:rados -p unfounddel_pool get new_7 - 2024-04-26T00:25:51.580 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_7 - 2024-04-26T00:25:51.657 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_7: (2) No such file or directory 2024-04-26T00:25:51.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.663 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_7 - 2024-04-26T00:25:51.664 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_7 - 2024-04-26T00:25:51.754 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_7: (2) No such file or directory 2024-04-26T00:25:51.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.759 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_7 - 2024-04-26T00:25:51.759 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_7 - 2024-04-26T00:25:51.831 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_7: (2) No such file or directory 2024-04-26T00:25:51.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.840 INFO:tasks.util.rados:rados -p unfounddel_pool get new_8 - 2024-04-26T00:25:51.840 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_8 - 2024-04-26T00:25:51.918 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_8: (2) No such file or directory 2024-04-26T00:25:51.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:51.924 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_8 - 2024-04-26T00:25:51.925 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_8 - 2024-04-26T00:25:52.005 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_8: (2) No such file or directory 2024-04-26T00:25:52.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:52.012 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_8 - 2024-04-26T00:25:52.012 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_8 - 2024-04-26T00:25:52.114 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_8: (2) No such file or directory 2024-04-26T00:25:52.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:52.119 INFO:tasks.util.rados:rados -p unfounddel_pool get new_9 - 2024-04-26T00:25:52.119 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_9 - 2024-04-26T00:25:52.197 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 32 20 52016 51996 6.34664 4.28125 0.00568468 0.0123028 2024-04-26T00:25:52.211 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/new_9: (2) No such file or directory 2024-04-26T00:25:52.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:52.218 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_9 - 2024-04-26T00:25:52.218 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_9 - 2024-04-26T00:25:52.308 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existed_9: (2) No such file or directory 2024-04-26T00:25:52.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:52.325 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_9 - 2024-04-26T00:25:52.325 DEBUG:teuthology.orchestra.run.smithi072:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_9 - 2024-04-26T00:25:52.368 INFO:teuthology.orchestra.run.smithi072.stderr:error getting unfounddel_pool/existing_9: (2) No such file or directory 2024-04-26T00:25:52.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:25:52.371 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-04-26T00:25:52.856 INFO:tasks.ceph.mon.a.smithi072.stderr:2024-04-26T00:25:52.874+0000 7ffb0b211640 -1 mon.a@0(leader).osd e29 definitely_dead 0 2024-04-26T00:25:53.198 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 33 20 52915 52895 6.26072 3.51172 0.0409317 0.012471 2024-04-26T00:25:53.258 INFO:teuthology.orchestra.run.smithi072.stderr:marked in osd.1. 2024-04-26T00:25:53.279 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-26T00:25:53.279 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-26T00:25:53.281 INFO:tasks.ceph.osd.1:Started 2024-04-26T00:25:53.282 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T00:25:53.400 INFO:teuthology.orchestra.run.smithi072.stderr:no valid command found; 10 closest matches: 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:0 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:1 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:2 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:abort 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:assert 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:config diff 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:config diff get 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:config get 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:config help [] 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:config set ... 2024-04-26T00:25:53.401 INFO:teuthology.orchestra.run.smithi072.stderr:admin_socket: invalid command 2024-04-26T00:25:53.403 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-26T00:25:53.403 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-04-26T00:25:53.878 INFO:tasks.ceph.osd.1.smithi072.stderr:2024-04-26T00:25:53.896+0000 7f564980e640 -1 Falling back to public interface 2024-04-26T00:25:54.198 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 34 20 53016 52996 6.08819 0.394531 0.0116958 0.0124785 2024-04-26T00:25:55.198 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 35 20 53016 52996 5.91423 0 - 0.0124785 2024-04-26T00:25:56.198 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 36 20 54289 54269 5.88806 2.48633 0.0129603 0.0132606 2024-04-26T00:25:57.198 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 37 20 55510 55490 5.85781 4.76953 0.00844129 0.0133271 2024-04-26T00:25:57.415 INFO:tasks.ceph.osd.1.smithi072.stderr:2024-04-26T00:25:57.433+0000 7f564980e640 -1 osd.1 23 log_to_monitors true 2024-04-26T00:25:58.198 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 38 20 56502 56482 5.80562 3.875 0.0144783 0.0134491 2024-04-26T00:25:58.314 INFO:tasks.ceph.osd.1.smithi072.stderr:2024-04-26T00:25:58.333+0000 7f5640ef7640 -1 osd.1 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T00:25:58.404 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-26T00:25:58.578 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:25:58.579 INFO:teuthology.orchestra.run.smithi072.stdout: "ops": [], 2024-04-26T00:25:58.579 INFO:teuthology.orchestra.run.smithi072.stdout: "num_ops": 0 2024-04-26T00:25:58.579 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:25:58.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.1 to be up 2024-04-26T00:25:58.593 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:25:59.106 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:25:59.106 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":33,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:25:58.304178+0000","last_up_change":"2024-04-26T00:25:07.935121+0000","last_in_change":"2024-04-26T00:25:53.266636+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.4900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-04-26T00:24:31.936865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":22,"up_thru":31,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":2078650556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":2078650556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":2078650556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":2078650556}]},"public_addr":"172.21.15.72:6800/2078650556","cluster_addr":"172.21.15.72:6801/2078650556","heartbeat_back_addr":"172.21.15.72:6803/2078650556","heartbeat_front_addr":"172.21.15.72:6802/2078650556","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":22,"down_at":24,"lost_at":24,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":1686523567}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":1686523567}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":1686523567}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":1686523567}]},"public_addr":"172.21.15.72:6808/1686523567","cluster_addr":"172.21.15.72:6809/1686523567","heartbeat_back_addr":"172.21.15.72:6811/1686523567","heartbeat_front_addr":"172.21.15.72:6810/1686523567","state":["exists"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":14,"up_from":28,"up_thru":31,"down_at":16,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":3995376740}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":3995376740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":3995376740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":3995376740}]},"public_addr":"172.21.15.72:6804/3995376740","cluster_addr":"172.21.15.72:6805/3995376740","heartbeat_back_addr":"172.21.15.72:6807/3995376740","heartbeat_front_addr":"172.21.15.72:6806/3995376740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":19},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":16}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.0","osds":[0,2]},{"pgid":"1.2","osds":[0,2]},{"pgid":"1.3","osds":[2,0]},{"pgid":"1.4","osds":[0,2]},{"pgid":"1.6","osds":[0,2]},{"pgid":"1.7","osds":[2,0]},{"pgid":"2.0","osds":[2,0]},{"pgid":"2.1","osds":[2,0]},{"pgid":"2.2","osds":[0,2]},{"pgid":"2.3","osds":[2,0]},{"pgid":"2.4","osds":[0,2]},{"pgid":"2.5","osds":[0,2]},{"pgid":"2.6","osds":[0,2]},{"pgid":"2.7","osds":[0,2]},{"pgid":"2.9","osds":[0,2]},{"pgid":"2.a","osds":[0,2]},{"pgid":"2.d","osds":[2,0]},{"pgid":"2.f","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:25:59.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 39 20 57615 57595 5.76823 4.34766 0.0193515 0.0135368 2024-04-26T00:26:00.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:26:00.218763+0000 min lat: 0.00233278 max lat: 2.04769 avg lat: 0.0135441 2024-04-26T00:26:00.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:26:00.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 40 20 57779 57759 5.64003 0.640625 0.00241717 0.0135441 2024-04-26T00:26:01.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 41 20 57779 57759 5.50246 0 - 0.0135441 2024-04-26T00:26:02.127 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:26:02.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 42 20 58060 58040 5.39758 0.548828 0.0324049 0.0144645 2024-04-26T00:26:02.673 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:26:02.674 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":36,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:26:01.347081+0000","last_up_change":"2024-04-26T00:25:59.320810+0000","last_in_change":"2024-04-26T00:25:53.266636+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-04-26T00:24:31.936865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":22,"up_thru":35,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":2078650556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":2078650556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":2078650556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":2078650556}]},"public_addr":"172.21.15.72:6800/2078650556","cluster_addr":"172.21.15.72:6801/2078650556","heartbeat_back_addr":"172.21.15.72:6803/2078650556","heartbeat_front_addr":"172.21.15.72:6802/2078650556","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":23,"up_from":34,"up_thru":35,"down_at":24,"lost_at":24,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":2570455510}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":2570455510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":2570455510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":2570455510}]},"public_addr":"172.21.15.72:6808/2570455510","cluster_addr":"172.21.15.72:6809/2570455510","heartbeat_back_addr":"172.21.15.72:6811/2570455510","heartbeat_front_addr":"172.21.15.72:6810/2570455510","state":["exists","up"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":14,"up_from":28,"up_thru":35,"down_at":16,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":3995376740}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":3995376740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":3995376740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":3995376740}]},"public_addr":"172.21.15.72:6804/3995376740","cluster_addr":"172.21.15.72:6805/3995376740","heartbeat_back_addr":"172.21.15.72:6807/3995376740","heartbeat_front_addr":"172.21.15.72:6806/3995376740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":19},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":16}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:26:02.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.1 is up 2024-04-26T00:26:02.702 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-04-26T00:26:02.702 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:26:03.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 43 20 58716 58696 5.33165 2.5625 0.0199814 0.0146431 2024-04-26T00:26:03.267 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:26:03.268 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:26:03.295 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-26T00:26:02.416196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":230110633,"num_objects":56182,"num_object_clones":0,"num_object_copies":112364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42010,"num_objects_unfound":0,"num_objects_dirty":56182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17184,"num_write_kb":68720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56808,"ondisk_log_size":56808,"up":30,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":734260,"kb_used_data":446300,"kb_used_omap":12,"kb_used_meta":287923,"kb_avail":282381260,"statfs":{"total":289910292480,"available":289158410240,"internally_reserved":0,"allocated":457011200,"data_stored":456393976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":294833260},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,6,12,7,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.091894"},"pg_stats":[{"pgid":"2.f","version":"32'3537","reported_seq":3586,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.908183+0000","last_change":"2024-04-26T00:25:55.308226+0000","last_active":"2024-04-26T00:25:57.908183+0000","last_peered":"2024-04-26T00:25:57.908183+0000","last_clean":"2024-04-26T00:25:57.908183+0000","last_became_active":"2024-04-26T00:25:55.303025+0000","last_became_peered":"2024-04-26T00:25:55.303025+0000","last_unstale":"2024-04-26T00:25:57.908183+0000","last_undegraded":"2024-04-26T00:25:57.908183+0000","last_fullsized":"2024-04-26T00:25:57.908183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3537,"log_dups_size":0,"ondisk_log_size":3537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:53.253796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14487552,"num_objects":3537,"num_object_clones":0,"num_object_copies":7074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3537,"num_objects_unfound":0,"num_objects_dirty":3537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3537,"num_write_kb":14148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3537}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'3690","reported_seq":3759,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:58.009482+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:25:58.009482+0000","last_peered":"2024-04-26T00:25:58.009482+0000","last_clean":"2024-04-26T00:25:58.009482+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:58.009482+0000","last_undegraded":"2024-04-26T00:25:58.009482+0000","last_fullsized":"2024-04-26T00:25:58.009482+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":3690,"log_dups_size":0,"ondisk_log_size":3690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15089664,"num_objects":3684,"num_object_clones":0,"num_object_copies":7368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":795,"num_write_kb":3180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'3499","reported_seq":3565,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:58.009552+0000","last_change":"2024-04-26T00:25:55.307329+0000","last_active":"2024-04-26T00:25:58.009552+0000","last_peered":"2024-04-26T00:25:58.009552+0000","last_clean":"2024-04-26T00:25:58.009552+0000","last_became_active":"2024-04-26T00:25:55.303174+0000","last_became_peered":"2024-04-26T00:25:55.303174+0000","last_unstale":"2024-04-26T00:25:58.009552+0000","last_undegraded":"2024-04-26T00:25:58.009552+0000","last_fullsized":"2024-04-26T00:25:58.009552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":3499,"log_dups_size":0,"ondisk_log_size":3499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:20:12.994155+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3495,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'3453","reported_seq":3478,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:58.009152+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:58.009152+0000","last_peered":"2024-04-26T00:25:58.009152+0000","last_clean":"2024-04-26T00:25:58.009152+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:58.009152+0000","last_undegraded":"2024-04-26T00:25:58.009152+0000","last_fullsized":"2024-04-26T00:25:58.009152+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3453,"log_dups_size":0,"ondisk_log_size":3453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143488,"num_objects":3453,"num_object_clones":0,"num_object_copies":6906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3453,"num_write_kb":13812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'3896","reported_seq":4099,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:57.908416+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:25:57.908416+0000","last_peered":"2024-04-26T00:25:57.908416+0000","last_clean":"2024-04-26T00:25:57.908416+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:57.908416+0000","last_undegraded":"2024-04-26T00:25:57.908416+0000","last_fullsized":"2024-04-26T00:25:57.908416+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":3896,"log_dups_size":0,"ondisk_log_size":3896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14614528,"num_objects":3568,"num_object_clones":0,"num_object_copies":7136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'3461","reported_seq":3562,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.942265+0000","last_change":"2024-04-26T00:25:55.317192+0000","last_active":"2024-04-26T00:25:57.942265+0000","last_peered":"2024-04-26T00:25:57.942265+0000","last_clean":"2024-04-26T00:25:57.942265+0000","last_became_active":"2024-04-26T00:25:55.313058+0000","last_became_peered":"2024-04-26T00:25:55.313058+0000","last_unstale":"2024-04-26T00:25:57.942265+0000","last_undegraded":"2024-04-26T00:25:57.942265+0000","last_fullsized":"2024-04-26T00:25:57.942265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":3461,"log_dups_size":0,"ondisk_log_size":3461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:31.963131+0000","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3456,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":636,"num_write_kb":2544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'3510","reported_seq":3613,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.927233+0000","last_change":"2024-04-26T00:25:55.308005+0000","last_active":"2024-04-26T00:25:57.927233+0000","last_peered":"2024-04-26T00:25:57.927233+0000","last_clean":"2024-04-26T00:25:57.927233+0000","last_became_active":"2024-04-26T00:25:55.303228+0000","last_became_peered":"2024-04-26T00:25:55.303228+0000","last_unstale":"2024-04-26T00:25:57.927233+0000","last_undegraded":"2024-04-26T00:25:57.927233+0000","last_fullsized":"2024-04-26T00:25:57.927233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":3510,"log_dups_size":0,"ondisk_log_size":3510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:54.385333+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":3504,"num_object_clones":0,"num_object_copies":7008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":3504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3504}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'3469","reported_seq":3652,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:57.699805+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:25:57.699805+0000","last_peered":"2024-04-26T00:25:57.699805+0000","last_clean":"2024-04-26T00:25:57.699805+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:57.699805+0000","last_undegraded":"2024-04-26T00:25:57.699805+0000","last_fullsized":"2024-04-26T00:25:57.699805+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":3469,"log_dups_size":0,"ondisk_log_size":3469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":667,"num_write_kb":2668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.305223+0000","last_change":"2024-04-26T00:25:55.305223+0000","last_active":"2024-04-26T00:25:55.305223+0000","last_peered":"2024-04-26T00:25:55.305223+0000","last_clean":"2024-04-26T00:25:55.305223+0000","last_became_active":"2024-04-26T00:25:55.304864+0000","last_became_peered":"2024-04-26T00:25:55.304864+0000","last_unstale":"2024-04-26T00:25:55.305223+0000","last_undegraded":"2024-04-26T00:25:55.305223+0000","last_fullsized":"2024-04-26T00:25:55.305223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:10.097014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'3611","reported_seq":3787,"reported_epoch":32,"state":"active+clean+remapped+scrubbing+deep","last_fresh":"2024-04-26T00:25:58.006296+0000","last_change":"2024-04-26T00:25:55.918781+0000","last_active":"2024-04-26T00:25:58.006296+0000","last_peered":"2024-04-26T00:25:58.006296+0000","last_clean":"2024-04-26T00:25:58.006296+0000","last_became_active":"2024-04-26T00:25:55.303312+0000","last_became_peered":"2024-04-26T00:25:55.303312+0000","last_unstale":"2024-04-26T00:25:58.006296+0000","last_undegraded":"2024-04-26T00:25:58.006296+0000","last_fullsized":"2024-04-26T00:25:58.006296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":624,"log_size":3611,"log_dups_size":0,"ondisk_log_size":3611,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13837254,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3380,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3383,"num_write_kb":13516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3380}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:55.296343+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:55.296343+0000","last_peered":"2024-04-26T00:25:55.296343+0000","last_clean":"2024-04-26T00:25:55.296343+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:55.296343+0000","last_undegraded":"2024-04-26T00:25:55.296343+0000","last_fullsized":"2024-04-26T00:25:55.296343+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'3524","reported_seq":3624,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.916017+0000","last_change":"2024-04-26T00:25:55.317078+0000","last_active":"2024-04-26T00:25:57.916017+0000","last_peered":"2024-04-26T00:25:57.916017+0000","last_clean":"2024-04-26T00:25:57.916017+0000","last_became_active":"2024-04-26T00:25:55.312460+0000","last_became_peered":"2024-04-26T00:25:55.312460+0000","last_unstale":"2024-04-26T00:25:57.916017+0000","last_undegraded":"2024-04-26T00:25:57.916017+0000","last_fullsized":"2024-04-26T00:25:57.916017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":3524,"log_dups_size":0,"ondisk_log_size":3524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:38:50.287926+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14401536,"num_objects":3516,"num_object_clones":0,"num_object_copies":7032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3516,"num_objects_unfound":0,"num_objects_dirty":3516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.316684+0000","last_change":"2024-04-26T00:25:55.316684+0000","last_active":"2024-04-26T00:25:55.316684+0000","last_peered":"2024-04-26T00:25:55.316684+0000","last_clean":"2024-04-26T00:25:55.316684+0000","last_became_active":"2024-04-26T00:25:55.312665+0000","last_became_peered":"2024-04-26T00:25:55.312665+0000","last_unstale":"2024-04-26T00:25:55.316684+0000","last_undegraded":"2024-04-26T00:25:55.316684+0000","last_fullsized":"2024-04-26T00:25:55.316684+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:28.355366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'3595","reported_seq":3671,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.994901+0000","last_change":"2024-04-26T00:25:55.309918+0000","last_active":"2024-04-26T00:25:57.994901+0000","last_peered":"2024-04-26T00:25:57.994901+0000","last_clean":"2024-04-26T00:25:57.994901+0000","last_became_active":"2024-04-26T00:25:55.302893+0000","last_became_peered":"2024-04-26T00:25:55.302893+0000","last_unstale":"2024-04-26T00:25:57.994901+0000","last_undegraded":"2024-04-26T00:25:57.994901+0000","last_fullsized":"2024-04-26T00:25:57.994901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:16:57.099585+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14704640,"num_objects":3590,"num_object_clones":0,"num_object_copies":7180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":3590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595,"num_write_kb":2380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3590}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.302616+0000","last_change":"2024-04-26T00:25:55.302616+0000","last_active":"2024-04-26T00:25:55.302616+0000","last_peered":"2024-04-26T00:25:55.302616+0000","last_clean":"2024-04-26T00:25:55.302616+0000","last_became_active":"2024-04-26T00:25:55.302205+0000","last_became_peered":"2024-04-26T00:25:55.302205+0000","last_unstale":"2024-04-26T00:25:55.302616+0000","last_undegraded":"2024-04-26T00:25:55.302616+0000","last_fullsized":"2024-04-26T00:25:55.302616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:54.490048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'3459","reported_seq":3534,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:58.003027+0000","last_change":"2024-04-26T00:25:55.306906+0000","last_active":"2024-04-26T00:25:58.003027+0000","last_peered":"2024-04-26T00:25:58.003027+0000","last_clean":"2024-04-26T00:25:58.003027+0000","last_became_active":"2024-04-26T00:25:55.302434+0000","last_became_peered":"2024-04-26T00:25:55.302434+0000","last_unstale":"2024-04-26T00:25:58.003027+0000","last_undegraded":"2024-04-26T00:25:58.003027+0000","last_fullsized":"2024-04-26T00:25:58.003027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":3459,"log_dups_size":0,"ondisk_log_size":3459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:34:21.708705+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143488,"num_objects":3453,"num_object_clones":0,"num_object_copies":6906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3453,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3453}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'3536","reported_seq":3635,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.918376+0000","last_change":"2024-04-26T00:25:55.308172+0000","last_active":"2024-04-26T00:25:57.918376+0000","last_peered":"2024-04-26T00:25:57.918376+0000","last_clean":"2024-04-26T00:25:57.918376+0000","last_became_active":"2024-04-26T00:25:55.303853+0000","last_became_peered":"2024-04-26T00:25:55.303853+0000","last_unstale":"2024-04-26T00:25:57.918376+0000","last_undegraded":"2024-04-26T00:25:57.918376+0000","last_fullsized":"2024-04-26T00:25:57.918376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:20.302256+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3534,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":457,"num_write_kb":1828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3534}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.313545+0000","last_change":"2024-04-26T00:25:55.313545+0000","last_active":"2024-04-26T00:25:55.313545+0000","last_peered":"2024-04-26T00:25:55.313545+0000","last_clean":"2024-04-26T00:25:55.313545+0000","last_became_active":"2024-04-26T00:25:55.313157+0000","last_became_peered":"2024-04-26T00:25:55.313157+0000","last_unstale":"2024-04-26T00:25:55.313545+0000","last_undegraded":"2024-04-26T00:25:55.313545+0000","last_fullsized":"2024-04-26T00:25:55.313545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:59:39.815158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'3463","reported_seq":3571,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.930887+0000","last_change":"2024-04-26T00:25:55.319361+0000","last_active":"2024-04-26T00:25:57.930887+0000","last_peered":"2024-04-26T00:25:57.930887+0000","last_clean":"2024-04-26T00:25:57.930887+0000","last_became_active":"2024-04-26T00:25:55.314701+0000","last_became_peered":"2024-04-26T00:25:55.314701+0000","last_unstale":"2024-04-26T00:25:57.930887+0000","last_undegraded":"2024-04-26T00:25:57.930887+0000","last_fullsized":"2024-04-26T00:25:57.930887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":3463,"log_dups_size":0,"ondisk_log_size":3463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:25:47.488320+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14151680,"num_objects":3455,"num_object_clones":0,"num_object_copies":6910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3455,"num_objects_unfound":0,"num_objects_dirty":3455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3455}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:55.300536+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:55.300536+0000","last_peered":"2024-04-26T00:25:55.300536+0000","last_clean":"2024-04-26T00:25:55.300536+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:55.300536+0000","last_undegraded":"2024-04-26T00:25:55.300536+0000","last_fullsized":"2024-04-26T00:25:55.300536+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'3567","reported_seq":3674,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.925245+0000","last_change":"2024-04-26T00:25:55.316688+0000","last_active":"2024-04-26T00:25:57.925245+0000","last_peered":"2024-04-26T00:25:57.925245+0000","last_clean":"2024-04-26T00:25:57.925245+0000","last_became_active":"2024-04-26T00:25:55.308850+0000","last_became_peered":"2024-04-26T00:25:55.308850+0000","last_unstale":"2024-04-26T00:25:57.925245+0000","last_undegraded":"2024-04-26T00:25:57.925245+0000","last_fullsized":"2024-04-26T00:25:57.925245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":3567,"log_dups_size":0,"ondisk_log_size":3567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:19:44.415135+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14594048,"num_objects":3563,"num_object_clones":0,"num_object_copies":7126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":3563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":263,"num_write_kb":1052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3563}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.316954+0000","last_change":"2024-04-26T00:25:55.316954+0000","last_active":"2024-04-26T00:25:55.316954+0000","last_peered":"2024-04-26T00:25:55.316954+0000","last_clean":"2024-04-26T00:25:55.316954+0000","last_became_active":"2024-04-26T00:25:55.314245+0000","last_became_peered":"2024-04-26T00:25:55.314245+0000","last_unstale":"2024-04-26T00:25:55.316954+0000","last_undegraded":"2024-04-26T00:25:55.316954+0000","last_fullsized":"2024-04-26T00:25:55.316954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:55.384317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.311324+0000","last_change":"2024-04-26T00:25:55.311324+0000","last_active":"2024-04-26T00:25:55.311324+0000","last_peered":"2024-04-26T00:25:55.311324+0000","last_clean":"2024-04-26T00:25:55.311324+0000","last_became_active":"2024-04-26T00:25:55.310992+0000","last_became_peered":"2024-04-26T00:25:55.310992+0000","last_unstale":"2024-04-26T00:25:55.311324+0000","last_undegraded":"2024-04-26T00:25:55.311324+0000","last_fullsized":"2024-04-26T00:25:55.311324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:07.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'3538","reported_seq":3643,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.938536+0000","last_change":"2024-04-26T00:25:55.319649+0000","last_active":"2024-04-26T00:25:57.938536+0000","last_peered":"2024-04-26T00:25:57.938536+0000","last_clean":"2024-04-26T00:25:57.938536+0000","last_became_active":"2024-04-26T00:25:55.313614+0000","last_became_peered":"2024-04-26T00:25:55.313614+0000","last_unstale":"2024-04-26T00:25:57.938536+0000","last_undegraded":"2024-04-26T00:25:57.938536+0000","last_fullsized":"2024-04-26T00:25:57.938536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":3538,"log_dups_size":0,"ondisk_log_size":3538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:48:51.427835+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14442979,"num_objects":3527,"num_object_clones":0,"num_object_copies":7054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":3527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":588,"num_write_kb":2352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3527}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":230110633,"num_objects":56182,"num_object_clones":0,"num_object_copies":112364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42010,"num_objects_unfound":0,"num_objects_dirty":56182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17184,"num_write_kb":68720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":460562432,"data_stored":460454042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56808,"ondisk_log_size":56808,"up":20,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":304,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":311296,"data_stored":70742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":28176079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280530,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351512,"kb_used_data":221328,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94020328,"statfs":{"total":96636764160,"available":96276815872,"internally_reserved":0,"allocated":226639872,"data_stored":226451537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":133295821},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084303,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354916,"kb_used_data":224668,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94016924,"statfs":{"total":96636764160,"available":96273330176,"internally_reserved":0,"allocated":230060032,"data_stored":229871697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":133361360},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,5,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":230051840,"data_stored":230048227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230395904,"data_stored":230392291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:26:03.296 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:26:03.700 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:26:03.703 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:26:03.724 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":84,"stamp":"2024-04-26T00:26:02.416196+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":230110633,"num_objects":56182,"num_object_clones":0,"num_object_copies":112364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42010,"num_objects_unfound":0,"num_objects_dirty":56182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17184,"num_write_kb":68720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56808,"ondisk_log_size":56808,"up":30,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":734260,"kb_used_data":446300,"kb_used_omap":12,"kb_used_meta":287923,"kb_avail":282381260,"statfs":{"total":289910292480,"available":289158410240,"internally_reserved":0,"allocated":457011200,"data_stored":456393976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":294833260},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,6,12,7,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.091894"},"pg_stats":[{"pgid":"2.f","version":"32'3537","reported_seq":3586,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.908183+0000","last_change":"2024-04-26T00:25:55.308226+0000","last_active":"2024-04-26T00:25:57.908183+0000","last_peered":"2024-04-26T00:25:57.908183+0000","last_clean":"2024-04-26T00:25:57.908183+0000","last_became_active":"2024-04-26T00:25:55.303025+0000","last_became_peered":"2024-04-26T00:25:55.303025+0000","last_unstale":"2024-04-26T00:25:57.908183+0000","last_undegraded":"2024-04-26T00:25:57.908183+0000","last_fullsized":"2024-04-26T00:25:57.908183+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3537,"log_dups_size":0,"ondisk_log_size":3537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:13:53.253796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14487552,"num_objects":3537,"num_object_clones":0,"num_object_copies":7074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3537,"num_objects_unfound":0,"num_objects_dirty":3537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3537,"num_write_kb":14148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3537}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"32'3690","reported_seq":3759,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:58.009482+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:25:58.009482+0000","last_peered":"2024-04-26T00:25:58.009482+0000","last_clean":"2024-04-26T00:25:58.009482+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:25:58.009482+0000","last_undegraded":"2024-04-26T00:25:58.009482+0000","last_fullsized":"2024-04-26T00:25:58.009482+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":3690,"log_dups_size":0,"ondisk_log_size":3690,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15089664,"num_objects":3684,"num_object_clones":0,"num_object_copies":7368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":795,"num_write_kb":3180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"32'3499","reported_seq":3565,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:58.009552+0000","last_change":"2024-04-26T00:25:55.307329+0000","last_active":"2024-04-26T00:25:58.009552+0000","last_peered":"2024-04-26T00:25:58.009552+0000","last_clean":"2024-04-26T00:25:58.009552+0000","last_became_active":"2024-04-26T00:25:55.303174+0000","last_became_peered":"2024-04-26T00:25:55.303174+0000","last_unstale":"2024-04-26T00:25:58.009552+0000","last_undegraded":"2024-04-26T00:25:58.009552+0000","last_fullsized":"2024-04-26T00:25:58.009552+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":3499,"log_dups_size":0,"ondisk_log_size":3499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:20:12.994155+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14315520,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3495,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":687,"num_write_kb":2748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3495}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"32'3453","reported_seq":3478,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:58.009152+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:25:58.009152+0000","last_peered":"2024-04-26T00:25:58.009152+0000","last_clean":"2024-04-26T00:25:58.009152+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:25:58.009152+0000","last_undegraded":"2024-04-26T00:25:58.009152+0000","last_fullsized":"2024-04-26T00:25:58.009152+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":3453,"log_dups_size":0,"ondisk_log_size":3453,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143488,"num_objects":3453,"num_object_clones":0,"num_object_copies":6906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3453,"num_write_kb":13812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"32'3896","reported_seq":4099,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:57.908416+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:25:57.908416+0000","last_peered":"2024-04-26T00:25:57.908416+0000","last_clean":"2024-04-26T00:25:57.908416+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:25:57.908416+0000","last_undegraded":"2024-04-26T00:25:57.908416+0000","last_fullsized":"2024-04-26T00:25:57.908416+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":3896,"log_dups_size":0,"ondisk_log_size":3896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14614528,"num_objects":3568,"num_object_clones":0,"num_object_copies":7136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"32'3461","reported_seq":3562,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.942265+0000","last_change":"2024-04-26T00:25:55.317192+0000","last_active":"2024-04-26T00:25:57.942265+0000","last_peered":"2024-04-26T00:25:57.942265+0000","last_clean":"2024-04-26T00:25:57.942265+0000","last_became_active":"2024-04-26T00:25:55.313058+0000","last_became_peered":"2024-04-26T00:25:55.313058+0000","last_unstale":"2024-04-26T00:25:57.942265+0000","last_undegraded":"2024-04-26T00:25:57.942265+0000","last_fullsized":"2024-04-26T00:25:57.942265+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":3461,"log_dups_size":0,"ondisk_log_size":3461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:26:31.963131+0000","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3456,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":636,"num_write_kb":2544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3456}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"32'3510","reported_seq":3613,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.927233+0000","last_change":"2024-04-26T00:25:55.308005+0000","last_active":"2024-04-26T00:25:57.927233+0000","last_peered":"2024-04-26T00:25:57.927233+0000","last_clean":"2024-04-26T00:25:57.927233+0000","last_became_active":"2024-04-26T00:25:55.303228+0000","last_became_peered":"2024-04-26T00:25:55.303228+0000","last_unstale":"2024-04-26T00:25:57.927233+0000","last_undegraded":"2024-04-26T00:25:57.927233+0000","last_fullsized":"2024-04-26T00:25:57.927233+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":3510,"log_dups_size":0,"ondisk_log_size":3510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:16:54.385333+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14352384,"num_objects":3504,"num_object_clones":0,"num_object_copies":7008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3504,"num_objects_unfound":0,"num_objects_dirty":3504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":712,"num_write_kb":2848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3504}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"32'3469","reported_seq":3652,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:57.699805+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:25:57.699805+0000","last_peered":"2024-04-26T00:25:57.699805+0000","last_clean":"2024-04-26T00:25:57.699805+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:25:57.699805+0000","last_undegraded":"2024-04-26T00:25:57.699805+0000","last_fullsized":"2024-04-26T00:25:57.699805+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":3469,"log_dups_size":0,"ondisk_log_size":3469,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14200832,"num_objects":3467,"num_object_clones":0,"num_object_copies":6934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":667,"num_write_kb":2668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.305223+0000","last_change":"2024-04-26T00:25:55.305223+0000","last_active":"2024-04-26T00:25:55.305223+0000","last_peered":"2024-04-26T00:25:55.305223+0000","last_clean":"2024-04-26T00:25:55.305223+0000","last_became_active":"2024-04-26T00:25:55.304864+0000","last_became_peered":"2024-04-26T00:25:55.304864+0000","last_unstale":"2024-04-26T00:25:55.305223+0000","last_undegraded":"2024-04-26T00:25:55.305223+0000","last_fullsized":"2024-04-26T00:25:55.305223+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:53:10.097014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"32'3611","reported_seq":3787,"reported_epoch":32,"state":"active+clean+remapped+scrubbing+deep","last_fresh":"2024-04-26T00:25:58.006296+0000","last_change":"2024-04-26T00:25:55.918781+0000","last_active":"2024-04-26T00:25:58.006296+0000","last_peered":"2024-04-26T00:25:58.006296+0000","last_clean":"2024-04-26T00:25:58.006296+0000","last_became_active":"2024-04-26T00:25:55.303312+0000","last_became_peered":"2024-04-26T00:25:55.303312+0000","last_unstale":"2024-04-26T00:25:58.006296+0000","last_undegraded":"2024-04-26T00:25:58.006296+0000","last_fullsized":"2024-04-26T00:25:58.006296+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":624,"log_size":3611,"log_dups_size":0,"ondisk_log_size":3611,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"deep scrubbing for 2s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13837254,"num_objects":3380,"num_object_clones":0,"num_object_copies":6760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3380,"num_objects_unfound":0,"num_objects_dirty":3380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3383,"num_write_kb":13516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3380}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:55.296343+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:25:55.296343+0000","last_peered":"2024-04-26T00:25:55.296343+0000","last_clean":"2024-04-26T00:25:55.296343+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:25:55.296343+0000","last_undegraded":"2024-04-26T00:25:55.296343+0000","last_fullsized":"2024-04-26T00:25:55.296343+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"32'3524","reported_seq":3624,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.916017+0000","last_change":"2024-04-26T00:25:55.317078+0000","last_active":"2024-04-26T00:25:57.916017+0000","last_peered":"2024-04-26T00:25:57.916017+0000","last_clean":"2024-04-26T00:25:57.916017+0000","last_became_active":"2024-04-26T00:25:55.312460+0000","last_became_peered":"2024-04-26T00:25:55.312460+0000","last_unstale":"2024-04-26T00:25:57.916017+0000","last_undegraded":"2024-04-26T00:25:57.916017+0000","last_fullsized":"2024-04-26T00:25:57.916017+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":3524,"log_dups_size":0,"ondisk_log_size":3524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:38:50.287926+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14401536,"num_objects":3516,"num_object_clones":0,"num_object_copies":7032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3516,"num_objects_unfound":0,"num_objects_dirty":3516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":305,"num_write_kb":1220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.316684+0000","last_change":"2024-04-26T00:25:55.316684+0000","last_active":"2024-04-26T00:25:55.316684+0000","last_peered":"2024-04-26T00:25:55.316684+0000","last_clean":"2024-04-26T00:25:55.316684+0000","last_became_active":"2024-04-26T00:25:55.312665+0000","last_became_peered":"2024-04-26T00:25:55.312665+0000","last_unstale":"2024-04-26T00:25:55.316684+0000","last_undegraded":"2024-04-26T00:25:55.316684+0000","last_fullsized":"2024-04-26T00:25:55.316684+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:19:28.355366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"32'3595","reported_seq":3671,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.994901+0000","last_change":"2024-04-26T00:25:55.309918+0000","last_active":"2024-04-26T00:25:57.994901+0000","last_peered":"2024-04-26T00:25:57.994901+0000","last_clean":"2024-04-26T00:25:57.994901+0000","last_became_active":"2024-04-26T00:25:55.302893+0000","last_became_peered":"2024-04-26T00:25:55.302893+0000","last_unstale":"2024-04-26T00:25:57.994901+0000","last_undegraded":"2024-04-26T00:25:57.994901+0000","last_fullsized":"2024-04-26T00:25:57.994901+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":3595,"log_dups_size":0,"ondisk_log_size":3595,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:16:57.099585+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14704640,"num_objects":3590,"num_object_clones":0,"num_object_copies":7180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3590,"num_objects_unfound":0,"num_objects_dirty":3590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595,"num_write_kb":2380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3590}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.302616+0000","last_change":"2024-04-26T00:25:55.302616+0000","last_active":"2024-04-26T00:25:55.302616+0000","last_peered":"2024-04-26T00:25:55.302616+0000","last_clean":"2024-04-26T00:25:55.302616+0000","last_became_active":"2024-04-26T00:25:55.302205+0000","last_became_peered":"2024-04-26T00:25:55.302205+0000","last_unstale":"2024-04-26T00:25:55.302616+0000","last_undegraded":"2024-04-26T00:25:55.302616+0000","last_fullsized":"2024-04-26T00:25:55.302616+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:07:54.490048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"32'3459","reported_seq":3534,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:58.003027+0000","last_change":"2024-04-26T00:25:55.306906+0000","last_active":"2024-04-26T00:25:58.003027+0000","last_peered":"2024-04-26T00:25:58.003027+0000","last_clean":"2024-04-26T00:25:58.003027+0000","last_became_active":"2024-04-26T00:25:55.302434+0000","last_became_peered":"2024-04-26T00:25:55.302434+0000","last_unstale":"2024-04-26T00:25:58.003027+0000","last_undegraded":"2024-04-26T00:25:58.003027+0000","last_fullsized":"2024-04-26T00:25:58.003027+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":3459,"log_dups_size":0,"ondisk_log_size":3459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:34:21.708705+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14143488,"num_objects":3453,"num_object_clones":0,"num_object_copies":6906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3453,"num_objects_unfound":0,"num_objects_dirty":3453,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":464,"num_write_kb":1856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3453}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"32'3536","reported_seq":3635,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.918376+0000","last_change":"2024-04-26T00:25:55.308172+0000","last_active":"2024-04-26T00:25:57.918376+0000","last_peered":"2024-04-26T00:25:57.918376+0000","last_clean":"2024-04-26T00:25:57.918376+0000","last_became_active":"2024-04-26T00:25:55.303853+0000","last_became_peered":"2024-04-26T00:25:55.303853+0000","last_unstale":"2024-04-26T00:25:57.918376+0000","last_undegraded":"2024-04-26T00:25:57.918376+0000","last_fullsized":"2024-04-26T00:25:57.918376+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":3536,"log_dups_size":0,"ondisk_log_size":3536,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:03:20.302256+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14475264,"num_objects":3534,"num_object_clones":0,"num_object_copies":7068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3534,"num_objects_unfound":0,"num_objects_dirty":3534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":457,"num_write_kb":1828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3534}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.313545+0000","last_change":"2024-04-26T00:25:55.313545+0000","last_active":"2024-04-26T00:25:55.313545+0000","last_peered":"2024-04-26T00:25:55.313545+0000","last_clean":"2024-04-26T00:25:55.313545+0000","last_became_active":"2024-04-26T00:25:55.313157+0000","last_became_peered":"2024-04-26T00:25:55.313157+0000","last_unstale":"2024-04-26T00:25:55.313545+0000","last_undegraded":"2024-04-26T00:25:55.313545+0000","last_fullsized":"2024-04-26T00:25:55.313545+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:59:39.815158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"32'3463","reported_seq":3571,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.930887+0000","last_change":"2024-04-26T00:25:55.319361+0000","last_active":"2024-04-26T00:25:57.930887+0000","last_peered":"2024-04-26T00:25:57.930887+0000","last_clean":"2024-04-26T00:25:57.930887+0000","last_became_active":"2024-04-26T00:25:55.314701+0000","last_became_peered":"2024-04-26T00:25:55.314701+0000","last_unstale":"2024-04-26T00:25:57.930887+0000","last_undegraded":"2024-04-26T00:25:57.930887+0000","last_fullsized":"2024-04-26T00:25:57.930887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":3463,"log_dups_size":0,"ondisk_log_size":3463,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:25:47.488320+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14151680,"num_objects":3455,"num_object_clones":0,"num_object_copies":6910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3455,"num_objects_unfound":0,"num_objects_dirty":3455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":345,"num_write_kb":1380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3455}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-04-26T00:25:55.300536+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:25:55.300536+0000","last_peered":"2024-04-26T00:25:55.300536+0000","last_clean":"2024-04-26T00:25:55.300536+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:25:55.300536+0000","last_undegraded":"2024-04-26T00:25:55.300536+0000","last_fullsized":"2024-04-26T00:25:55.300536+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"32'3567","reported_seq":3674,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.925245+0000","last_change":"2024-04-26T00:25:55.316688+0000","last_active":"2024-04-26T00:25:57.925245+0000","last_peered":"2024-04-26T00:25:57.925245+0000","last_clean":"2024-04-26T00:25:57.925245+0000","last_became_active":"2024-04-26T00:25:55.308850+0000","last_became_peered":"2024-04-26T00:25:55.308850+0000","last_unstale":"2024-04-26T00:25:57.925245+0000","last_undegraded":"2024-04-26T00:25:57.925245+0000","last_fullsized":"2024-04-26T00:25:57.925245+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":3567,"log_dups_size":0,"ondisk_log_size":3567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:19:44.415135+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14594048,"num_objects":3563,"num_object_clones":0,"num_object_copies":7126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3563,"num_objects_unfound":0,"num_objects_dirty":3563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":263,"num_write_kb":1052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3563}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.316954+0000","last_change":"2024-04-26T00:25:55.316954+0000","last_active":"2024-04-26T00:25:55.316954+0000","last_peered":"2024-04-26T00:25:55.316954+0000","last_clean":"2024-04-26T00:25:55.316954+0000","last_became_active":"2024-04-26T00:25:55.314245+0000","last_became_peered":"2024-04-26T00:25:55.314245+0000","last_unstale":"2024-04-26T00:25:55.316954+0000","last_undegraded":"2024-04-26T00:25:55.316954+0000","last_fullsized":"2024-04-26T00:25:55.316954+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:32:55.384317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":23,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:55.311324+0000","last_change":"2024-04-26T00:25:55.311324+0000","last_active":"2024-04-26T00:25:55.311324+0000","last_peered":"2024-04-26T00:25:55.311324+0000","last_clean":"2024-04-26T00:25:55.311324+0000","last_became_active":"2024-04-26T00:25:55.310992+0000","last_became_peered":"2024-04-26T00:25:55.310992+0000","last_unstale":"2024-04-26T00:25:55.311324+0000","last_undegraded":"2024-04-26T00:25:55.311324+0000","last_fullsized":"2024-04-26T00:25:55.311324+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:55:07.888905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"32'3538","reported_seq":3643,"reported_epoch":32,"state":"active+clean+remapped","last_fresh":"2024-04-26T00:25:57.938536+0000","last_change":"2024-04-26T00:25:55.319649+0000","last_active":"2024-04-26T00:25:57.938536+0000","last_peered":"2024-04-26T00:25:57.938536+0000","last_clean":"2024-04-26T00:25:57.938536+0000","last_became_active":"2024-04-26T00:25:55.313614+0000","last_became_peered":"2024-04-26T00:25:55.313614+0000","last_unstale":"2024-04-26T00:25:57.938536+0000","last_undegraded":"2024-04-26T00:25:57.938536+0000","last_fullsized":"2024-04-26T00:25:57.938536+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":3538,"log_dups_size":0,"ondisk_log_size":3538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:48:51.427835+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14442979,"num_objects":3527,"num_object_clones":0,"num_object_copies":7054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3527,"num_objects_unfound":0,"num_objects_dirty":3527,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":588,"num_write_kb":2352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":3527}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":230110633,"num_objects":56182,"num_object_clones":0,"num_object_copies":112364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":42010,"num_objects_unfound":0,"num_objects_dirty":56182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17184,"num_write_kb":68720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":460562432,"data_stored":460454042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56808,"ondisk_log_size":56808,"up":20,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":10,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":304,"kb_used_omap":4,"kb_used_meta":27515,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":311296,"data_stored":70742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":28176079},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280530,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":351512,"kb_used_data":221328,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94020328,"statfs":{"total":96636764160,"available":96276815872,"internally_reserved":0,"allocated":226639872,"data_stored":226451537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":133295821},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,7,3,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084303,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354916,"kb_used_data":224668,"kb_used_omap":4,"kb_used_meta":130235,"kb_avail":94016924,"statfs":{"total":96636764160,"available":96273330176,"internally_reserved":0,"allocated":230060032,"data_stored":229871697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":133361360},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,5,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":230051840,"data_stored":230048227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":13524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230395904,"data_stored":230392291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:26:03.725 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-04-26T00:26:04.199 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 44 20 59324 59304 5.26445 2.375 0.013634 0.0148313 2024-04-26T00:26:05.202 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 45 20 60079 60059 5.21265 2.94922 0.0405589 0.0149789 2024-04-26T00:26:06.202 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 46 20 60825 60805 5.16268 2.91406 0.0203392 0.0151225 2024-04-26T00:26:07.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 47 20 61578 61558 5.11542 2.94141 0.00800601 0.0152628 2024-04-26T00:26:08.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 48 20 62331 62311 5.07012 2.94141 0.00943909 0.0154 2024-04-26T00:26:09.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 49 20 62549 62529 4.98403 0.851562 0.0249376 0.0154425 2024-04-26T00:26:10.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 50 19 63065 63046 4.92474 2.01953 0.0136484 0.0158557 2024-04-26T00:26:11.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 51 20 63815 63795 4.88554 2.92578 0.0177542 0.0159823 2024-04-26T00:26:12.203 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 52 20 64551 64531 4.84688 2.875 0.0270006 0.0161107 2024-04-26T00:26:13.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 53 20 65298 65278 4.81028 2.91797 0.0250738 0.016233 2024-04-26T00:26:14.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 54 20 65883 65863 4.76352 2.28516 0.062104 0.0163917 2024-04-26T00:26:15.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 55 20 66598 66578 4.72769 2.79297 0.0126711 0.0165159 2024-04-26T00:26:16.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 56 20 67322 67302 4.69377 2.82812 0.0196667 0.0166337 2024-04-26T00:26:17.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 57 20 68054 68034 4.66159 2.85938 0.0161835 0.0167521 2024-04-26T00:26:18.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 58 20 68768 68748 4.6293 2.78906 0.0270676 0.0168676 2024-04-26T00:26:19.206 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 59 20 69437 69417 4.59513 2.61328 0.0218397 0.0169953 2024-04-26T00:26:20.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:26:20.225664+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.0171063 2024-04-26T00:26:20.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:26:20.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 60 20 70153 70133 4.56516 2.79688 0.023258 0.0171063 2024-04-26T00:26:21.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 61 20 70806 70786 4.53213 2.55078 0.0290305 0.01723 2024-04-26T00:26:22.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 62 20 71512 71492 4.50351 2.75781 0.0587079 0.0173381 2024-04-26T00:26:23.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 63 20 72217 72197 4.47574 2.75391 0.0407691 0.0174457 2024-04-26T00:26:24.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 64 20 72817 72797 4.44243 2.34375 0.0158788 0.0175783 2024-04-26T00:26:25.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 65 20 73526 73506 4.41669 2.76953 0.0120208 0.0176805 2024-04-26T00:26:26.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 66 20 74220 74200 4.39084 2.71094 0.00761929 0.0177842 2024-04-26T00:26:27.207 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 67 20 74900 74880 4.36495 2.65625 0.0058242 0.017888 2024-04-26T00:26:28.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 68 19 75584 75565 4.34011 2.67578 0.0546054 0.0179936 2024-04-26T00:26:29.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 69 20 76239 76219 4.31424 2.55469 0.015786 0.0180972 2024-04-26T00:26:30.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 70 20 76930 76910 4.29116 2.69922 0.0289407 0.0181973 2024-04-26T00:26:31.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 71 20 77608 77588 4.26802 2.64844 0.0606291 0.0182963 2024-04-26T00:26:32.208 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 72 20 78305 78285 4.24656 2.72266 0.0213124 0.0183895 2024-04-26T00:26:33.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 73 20 78985 78965 4.22457 2.65625 0.0182894 0.0184854 2024-04-26T00:26:34.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 74 20 79583 79563 4.19905 2.33594 0.0264136 0.0185963 2024-04-26T00:26:35.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 75 20 80256 80236 4.17812 2.62891 0.0268463 0.0186891 2024-04-26T00:26:36.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 76 20 80921 80901 4.15732 2.59766 0.0367636 0.0187848 2024-04-26T00:26:37.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 77 20 81587 81567 4.13712 2.60156 0.0275394 0.0188769 2024-04-26T00:26:38.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 78 20 82259 82239 4.11773 2.625 0.0322911 0.0189657 2024-04-26T00:26:39.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 79 20 82928 82908 4.09868 2.61328 0.0206594 0.0190545 2024-04-26T00:26:40.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:26:40.230763+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.0191571 2024-04-26T00:26:40.213 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:26:40.213 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 80 20 83524 83504 4.07655 2.32812 0.0386126 0.0191571 2024-04-26T00:26:41.212 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 81 20 84070 84050 4.05256 2.13281 0.0197962 0.0192703 2024-04-26T00:26:42.213 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 82 20 84736 84716 4.03486 2.60156 0.0574076 0.0193532 2024-04-26T00:26:43.213 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 83 20 85387 85367 4.01688 2.54297 0.028593 0.0194395 2024-04-26T00:26:44.213 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 84 20 85979 85959 3.99659 2.3125 0.0318135 0.0195401 2024-04-26T00:26:45.213 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 85 20 86630 86610 3.97949 2.54297 0.0115424 0.0196225 2024-04-26T00:26:46.213 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 86 20 87291 87271 3.96324 2.58203 0.0228485 0.0197047 2024-04-26T00:26:47.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 87 20 87944 87924 3.94695 2.55078 0.0329793 0.0197841 2024-04-26T00:26:48.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 88 20 88599 88579 3.93117 2.55859 0.0194676 0.019866 2024-04-26T00:26:49.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 89 20 89248 89228 3.91549 2.53516 0.0114491 0.019944 2024-04-26T00:26:50.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 90 20 89796 89776 3.89577 2.14062 0.211732 0.0200218 2024-04-26T00:26:51.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 91 20 90448 90428 3.88094 2.54688 0.0316809 0.0201233 2024-04-26T00:26:52.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 92 20 91098 91078 3.86636 2.53906 0.0309936 0.0201981 2024-04-26T00:26:53.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 93 20 91725 91705 3.85112 2.44922 0.0338224 0.0202769 2024-04-26T00:26:54.215 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 94 20 92344 92324 3.83587 2.41797 0.0275296 0.0203603 2024-04-26T00:26:55.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 95 20 92989 92969 3.82201 2.51953 0.0337551 0.0204326 2024-04-26T00:26:56.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 96 20 93621 93601 3.80792 2.46875 0.0111323 0.0205076 2024-04-26T00:26:57.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 97 20 94250 94230 3.79399 2.45703 0.0361352 0.0205837 2024-04-26T00:26:58.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 98 20 94880 94860 3.78039 2.46094 0.0176402 0.0206575 2024-04-26T00:26:59.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 99 20 95403 95383 3.76284 2.04297 0.0492135 0.0207537 2024-04-26T00:27:00.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:27:00.233663+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.020826 2024-04-26T00:27:00.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:27:00.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 100 20 96042 96022 3.75017 2.49609 0.0534728 0.020826 2024-04-26T00:27:01.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 101 20 96673 96653 3.73744 2.46484 0.0520125 0.0208952 2024-04-26T00:27:02.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 102 20 97296 97276 3.72466 2.43359 0.0325417 0.020968 2024-04-26T00:27:03.216 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 103 20 97920 97900 3.71216 2.4375 0.0180014 0.0210383 2024-04-26T00:27:04.217 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 104 20 98495 98475 3.69806 2.24609 0.0410765 0.0211187 2024-04-26T00:27:05.217 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 105 20 99117 99097 3.68598 2.42969 0.0238101 0.0211878 2024-04-26T00:27:06.217 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 106 20 99746 99726 3.67439 2.45703 0.055564 0.0212534 2024-04-26T00:27:07.217 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 107 20 100269 100249 3.65914 2.04297 0.0138766 0.0213436 2024-04-26T00:27:08.217 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 108 20 100891 100871 3.64776 2.42969 0.0373421 0.0214084 2024-04-26T00:27:09.217 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 109 20 101449 101429 3.63429 2.17969 0.0174293 0.0214899 2024-04-26T00:27:10.217 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 110 20 102063 102043 3.62305 2.39844 0.0235529 0.0215555 2024-04-26T00:27:11.217 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 111 20 102664 102644 3.61156 2.34766 0.0531618 0.0216244 2024-04-26T00:27:12.218 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 112 20 103287 103267 3.60104 2.43359 0.0364792 0.0216871 2024-04-26T00:27:13.218 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 113 20 103900 103880 3.59036 2.39453 0.0570665 0.021752 2024-04-26T00:27:14.218 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 114 20 104441 104421 3.57741 2.11328 0.0200807 0.021831 2024-04-26T00:27:15.218 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 115 20 105057 105037 3.56722 2.40625 0.0436786 0.0218935 2024-04-26T00:27:16.218 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 116 20 105670 105650 3.55711 2.39453 0.0165991 0.0219542 2024-04-26T00:27:17.218 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 117 20 106271 106251 3.54677 2.34766 0.0108131 0.0220193 2024-04-26T00:27:18.218 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 118 20 106882 106862 3.53694 2.38672 0.0433708 0.0220812 2024-04-26T00:27:19.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 119 20 107483 107463 3.52694 2.34766 0.0260467 0.0221446 2024-04-26T00:27:20.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:27:20.235457+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.022206 2024-04-26T00:27:20.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:27:20.219 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 120 20 108081 108061 3.51702 2.33594 0.0535589 0.022206 2024-04-26T00:27:21.220 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 121 20 108691 108671 3.5076 2.38281 0.0439368 0.0222654 2024-04-26T00:27:22.221 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 122 20 109293 109273 3.49812 2.35156 0.0534992 0.0223272 2024-04-26T00:27:23.221 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 123 20 109892 109872 3.4887 2.33984 0.0246289 0.0223858 2024-04-26T00:27:24.221 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 124 20 110491 110471 3.47944 2.33984 0.0333894 0.0224455 2024-04-26T00:27:25.221 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 125 20 111088 111068 3.47026 2.33203 0.0186916 0.0225054 2024-04-26T00:27:26.221 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 126 20 111683 111663 3.46116 2.32422 0.0316829 0.022565 2024-04-26T00:27:27.221 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 127 20 112287 112267 3.45248 2.35938 0.0189727 0.0226222 2024-04-26T00:27:28.221 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 128 20 112869 112849 3.44327 2.27344 0.0290185 0.0226822 2024-04-26T00:27:29.221 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 129 19 113386 113367 3.43227 2.02344 0.0225515 0.0227545 2024-04-26T00:27:30.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 130 20 113985 113965 3.42383 2.33594 0.0415511 0.0228107 2024-04-26T00:27:31.222 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 131 20 114571 114551 3.41517 2.28906 0.0220138 0.0228677 2024-04-26T00:27:32.222 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 132 20 115160 115140 3.40673 2.30078 0.0249483 0.0229256 2024-04-26T00:27:33.222 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 133 20 115755 115735 3.39859 2.32422 0.02768 0.0229793 2024-04-26T00:27:34.222 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 134 20 116215 116195 3.38663 1.79688 0.0231481 0.023062 2024-04-26T00:27:35.222 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 135 20 116793 116773 3.37827 2.25781 0.043836 0.0231178 2024-04-26T00:27:36.222 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 136 20 117384 117364 3.3704 2.30859 0.013854 0.0231732 2024-04-26T00:27:37.222 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 137 20 117952 117932 3.362 2.21875 0.0165701 0.0232275 2024-04-26T00:27:38.222 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 138 20 118183 118163 3.34417 0.902344 0.0305589 0.023354 2024-04-26T00:27:39.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 139 20 118677 118657 3.334 1.92969 0.0353699 0.0234248 2024-04-26T00:27:40.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:27:40.238759+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.0234782 2024-04-26T00:27:40.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:27:40.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 140 20 119262 119242 3.32651 2.28516 0.0313077 0.0234782 2024-04-26T00:27:41.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 141 20 119838 119818 3.31887 2.25 0.0159573 0.0235319 2024-04-26T00:27:42.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 142 20 120416 120396 3.3114 2.25781 0.0567938 0.0235853 2024-04-26T00:27:43.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 143 20 120980 120960 3.30365 2.20312 0.0131277 0.0236399 2024-04-26T00:27:44.223 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 144 20 121456 121436 3.29362 1.85938 0.0389162 0.0237116 2024-04-26T00:27:45.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 145 20 122018 121998 3.28604 2.19531 0.0727567 0.0237667 2024-04-26T00:27:46.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 146 20 122595 122575 3.27897 2.25391 0.0398655 0.0238188 2024-04-26T00:27:47.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 147 20 123173 123153 3.27202 2.25781 0.0242047 0.0238706 2024-04-26T00:27:48.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 148 20 123735 123715 3.26474 2.19531 0.0223168 0.0239232 2024-04-26T00:27:49.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 149 20 124299 124279 3.25762 2.20312 0.0225905 0.0239767 2024-04-26T00:27:50.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 150 20 124863 124843 3.25059 2.20312 0.0194303 0.0240267 2024-04-26T00:27:51.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 151 20 125427 125407 3.24365 2.20312 0.0642261 0.0240786 2024-04-26T00:27:52.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 152 20 125995 125975 3.23691 2.21875 0.0509174 0.0241278 2024-04-26T00:27:53.224 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 153 20 126543 126523 3.22974 2.14062 0.0280046 0.0241807 2024-04-26T00:27:54.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 154 20 127049 127029 3.2216 1.97656 0.042539 0.0242442 2024-04-26T00:27:55.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 155 20 127605 127585 3.21483 2.17188 0.0103369 0.0242943 2024-04-26T00:27:56.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 156 20 128160 128140 3.20812 2.16797 0.058342 0.0243447 2024-04-26T00:27:57.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 157 20 128508 128488 3.19634 1.35938 0.0240062 0.0244303 2024-04-26T00:27:58.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 158 20 129061 129041 3.18979 2.16016 0.0269563 0.0244842 2024-04-26T00:27:59.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 159 20 129564 129544 3.18208 1.96484 0.0250798 0.0245449 2024-04-26T00:28:00.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:28:00.240586+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.02461 2024-04-26T00:28:00.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:28:00.226 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 160 20 130027 130007 3.1735 1.80859 0.0353837 0.02461 2024-04-26T00:28:01.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 161 20 130519 130499 3.16572 1.92188 0.0183837 0.024671 2024-04-26T00:28:02.225 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 162 20 131059 131039 3.1592 2.10938 0.0368604 0.0247225 2024-04-26T00:28:03.227 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 163 20 131612 131592 3.15305 2.16016 0.060877 0.02477 2024-04-26T00:28:04.227 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 164 20 132144 132124 3.14649 2.07812 0.0139041 0.0248215 2024-04-26T00:28:05.227 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 165 20 132701 132681 3.14061 2.17578 0.0341037 0.0248697 2024-04-26T00:28:06.227 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 166 20 133240 133220 3.13437 2.10547 0.0323868 0.0249183 2024-04-26T00:28:07.227 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 167 20 133787 133767 3.1284 2.13672 0.0234439 0.0249651 2024-04-26T00:28:08.227 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 168 20 134331 134311 3.12243 2.125 0.0692802 0.0250123 2024-04-26T00:28:09.229 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 169 20 134875 134855 3.11652 2.125 0.0203228 0.0250602 2024-04-26T00:28:10.228 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 170 20 135426 135406 3.11085 2.15234 0.0288552 0.0251073 2024-04-26T00:28:11.228 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 171 20 135922 135902 3.10399 1.9375 0.00603126 0.0251476 2024-04-26T00:28:12.228 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 172 20 136440 136420 3.09771 2.02344 0.0141801 0.0252115 2024-04-26T00:28:13.228 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 173 20 136970 136950 3.09177 2.07031 0.0301285 0.0252611 2024-04-26T00:28:14.228 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 174 20 137463 137443 3.08507 1.92578 0.0343953 0.0253154 2024-04-26T00:28:15.228 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 175 20 138010 137990 3.07965 2.13672 0.037825 0.0253601 2024-04-26T00:28:16.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 176 20 138549 138529 3.07404 2.10547 0.0332588 0.0254073 2024-04-26T00:28:17.232 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 177 20 139085 139065 3.0685 2.09375 0.0278063 0.0254529 2024-04-26T00:28:18.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 178 20 139607 139587 3.06272 2.03906 0.0526289 0.0255005 2024-04-26T00:28:19.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 179 20 140131 140111 3.05704 2.04688 0.0488671 0.0255479 2024-04-26T00:28:20.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:28:20.247527+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.0255955 2024-04-26T00:28:20.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:28:20.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 180 20 140658 140638 3.05149 2.05859 0.0407689 0.0255955 2024-04-26T00:28:21.233 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 181 20 141189 141169 3.04609 2.07422 0.0555821 0.0256402 2024-04-26T00:28:22.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 182 20 141717 141697 3.04066 2.0625 0.0888455 0.0256856 2024-04-26T00:28:23.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 183 20 142242 142222 3.03525 2.05078 0.0250281 0.0257337 2024-04-26T00:28:24.235 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 184 20 142758 142738 3.0297 2.01562 0.0530167 0.02578 2024-04-26T00:28:25.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 185 20 143282 143262 3.02439 2.04688 0.0419147 0.0258254 2024-04-26T00:28:26.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 186 20 143797 143777 3.01895 2.01172 0.0270415 0.025871 2024-04-26T00:28:27.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 187 20 144297 144277 3.01325 1.95312 0.00533413 0.0259155 2024-04-26T00:28:28.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 188 20 144745 144725 3.00653 1.75 0.0281119 0.0259784 2024-04-26T00:28:29.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 189 20 145214 145194 3.00031 1.83203 0.0472164 0.0260323 2024-04-26T00:28:30.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 190 20 145732 145712 2.99517 2.02344 0.0308455 0.026076 2024-04-26T00:28:31.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 191 20 146245 146225 2.98998 2.00391 0.0310718 0.0261216 2024-04-26T00:28:32.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 192 20 146765 146745 2.98499 2.03125 0.0332318 0.0261656 2024-04-26T00:28:33.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 193 20 147285 147265 2.98005 2.03125 0.0518846 0.0262089 2024-04-26T00:28:34.236 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 194 20 147744 147724 2.97393 1.79297 0.0122059 0.0262623 2024-04-26T00:28:35.237 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 195 20 148266 148246 2.96913 2.03906 0.0632579 0.0263071 2024-04-26T00:28:36.237 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 196 20 148721 148701 2.96305 1.77734 0.106422 0.0263514 2024-04-26T00:28:37.237 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 197 20 149237 149217 2.95824 2.01562 0.108615 0.0264013 2024-04-26T00:28:38.238 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 198 20 149667 149647 2.95176 1.67969 0.0126541 0.0264598 2024-04-26T00:28:39.238 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 199 20 150183 150163 2.94706 2.01562 0.0438464 0.0265031 2024-04-26T00:28:40.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:28:40.252643+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.0265449 2024-04-26T00:28:40.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:28:40.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 200 20 150699 150679 2.9424 2.01562 0.0258054 0.0265449 2024-04-26T00:28:41.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 201 20 151188 151168 2.93726 1.91016 0.00557033 0.0265833 2024-04-26T00:28:42.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 202 20 151634 151614 2.93135 1.74219 0.0427509 0.0266452 2024-04-26T00:28:43.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 203 20 152140 152120 2.92664 1.97656 0.0511761 0.0266877 2024-04-26T00:28:44.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 204 20 152649 152629 2.92204 1.98828 0.0325388 0.0267295 2024-04-26T00:28:45.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 205 20 153163 153143 2.91758 2.00781 0.03009 0.0267707 2024-04-26T00:28:46.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 206 20 153661 153641 2.91286 1.94531 0.0512109 0.0268134 2024-04-26T00:28:47.239 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 207 20 154166 154146 2.90832 1.97266 0.0334277 0.0268561 2024-04-26T00:28:48.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 208 20 154634 154614 2.90313 1.82812 0.0317913 0.0269039 2024-04-26T00:28:49.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 209 20 155124 155104 2.89839 1.91406 0.040713 0.0269469 2024-04-26T00:28:50.240 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 210 20 155625 155605 2.89391 1.95703 0.0155659 0.0269879 2024-04-26T00:28:51.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 211 20 156138 156118 2.88964 2.00391 0.0451513 0.0270297 2024-04-26T00:28:52.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 212 20 156619 156599 2.88488 1.87891 0.0323301 0.0270736 2024-04-26T00:28:53.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 213 20 157116 157096 2.88045 1.94141 0.0290172 0.0271159 2024-04-26T00:28:54.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 214 20 157546 157526 2.87483 1.67969 0.0410707 0.0271689 2024-04-26T00:28:55.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 215 20 157980 157960 2.86935 1.69531 0.0322905 0.0272203 2024-04-26T00:28:56.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 216 20 158432 158412 2.86424 1.76562 0.0320325 0.0272694 2024-04-26T00:28:57.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 217 20 158931 158911 2.86002 1.94922 0.0526346 0.0273093 2024-04-26T00:28:58.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 218 20 159418 159398 2.85563 1.90234 0.0479678 0.0273512 2024-04-26T00:28:59.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 219 20 159910 159890 2.85136 1.92188 0.0382697 0.0273921 2024-04-26T00:29:00.244 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:2024-04-26T00:29:00.257969+0000 min lat: 0.00233278 max lat: 2.55536 avg lat: 0.0274362 2024-04-26T00:29:00.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-04-26T00:29:00.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 220 20 160379 160359 2.84673 1.83203 0.0627542 0.0274362 2024-04-26T00:29:01.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 221 20 160872 160852 2.84256 1.92578 0.0236179 0.0274769 2024-04-26T00:29:02.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 222 20 161358 161338 2.83831 1.89844 0.0229519 0.0275177 2024-04-26T00:29:03.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 223 20 161812 161792 2.83353 1.77344 0.031441 0.0275643 2024-04-26T00:29:04.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 224 20 162311 162291 2.82959 1.94922 0.0379079 0.0276026 2024-04-26T00:29:05.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 225 20 162805 162785 2.82559 1.92969 0.0243336 0.0276417 2024-04-26T00:29:06.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 226 20 163292 163272 2.8215 1.90234 0.0306941 0.0276819 2024-04-26T00:29:07.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 227 20 163792 163772 2.81768 1.95312 0.0277241 0.02772 2024-04-26T00:29:08.245 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 228 20 164288 164268 2.81381 1.9375 0.0182257 0.0277575 2024-04-26T00:29:09.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 229 20 164781 164761 2.80994 1.92578 0.0280166 0.0277954 2024-04-26T00:29:10.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 230 20 165279 165259 2.80618 1.94531 0.0424055 0.0278333 2024-04-26T00:29:11.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 231 20 165768 165748 2.8023 1.91016 0.107037 0.0278706 2024-04-26T00:29:12.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 232 20 166257 166237 2.79845 1.91016 0.0334255 0.0279103 2024-04-26T00:29:13.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 233 20 166748 166728 2.79467 1.91797 0.0329576 0.0279481 2024-04-26T00:29:14.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 234 20 167242 167222 2.79098 1.92969 0.025246 0.0279848 2024-04-26T00:29:15.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 235 20 167704 167684 2.78678 1.80469 0.0237048 0.0280267 2024-04-26T00:29:16.246 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 236 19 168192 168173 2.78306 1.91016 0.0510465 0.0280649 2024-04-26T00:29:17.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 237 20 168689 168669 2.7795 1.9375 0.0295739 0.0281004 2024-04-26T00:29:18.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 238 20 169185 169165 2.77596 1.9375 0.0467135 0.0281367 2024-04-26T00:29:19.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout: 239 20 169673 169653 2.77232 1.90625 0.0509254 0.028172 2024-04-26T00:29:20.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Total time run: 240.02 2024-04-26T00:29:20.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Total writes made: 170084 2024-04-26T00:29:20.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Write size: 4096 2024-04-26T00:29:20.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Object size: 4096 2024-04-26T00:29:20.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Bandwidth (MB/sec): 2.76806 2024-04-26T00:29:20.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Stddev Bandwidth: 1.68002 2024-04-26T00:29:20.247 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Max bandwidth (MB/sec): 10.4453 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Min bandwidth (MB/sec): 0 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Average IOPS: 708 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Stddev IOPS: 430.088 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Max IOPS: 2674 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Min IOPS: 0 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Average Latency(s): 0.0282187 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Stddev Latency(s): 0.0437582 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Max latency(s): 2.55536 2024-04-26T00:29:20.248 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Min latency(s): 0.00233278 2024-04-26T00:29:20.258 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Cleaning up (deleting benchmark objects) 2024-04-26T00:34:57.081 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Removed 170084 objects 2024-04-26T00:34:57.081 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi072.stdout:Clean up completed and total clean up time :336.814 2024-04-26T00:34:57.090 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-04-26T00:34:57.090 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:34:57.878 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:34:57.879 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:34:57.899 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-04-26T00:34:56.570957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8623553,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":299069,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17986,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540896,"kb_used_data":193660,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281574624,"statfs":{"total":289910292480,"available":288332414976,"internally_reserved":0,"allocated":198307840,"data_stored":197508800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,8,7,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25956352,"num_objects":-6337,"num_object_clones":0,"num_object_copies":-12674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6337,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002576"},"pg_stats":[{"pgid":"2.f","version":"44'21096","reported_seq":21748,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.183152+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:52.183152+0000","last_peered":"2024-04-26T00:34:52.183152+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:52.183152+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":671744,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21096,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21614","reported_seq":21711,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.945950+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:54.945950+0000","last_peered":"2024-04-26T00:34:54.945950+0000","last_clean":"2024-04-26T00:34:54.945950+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:54.945950+0000","last_undegraded":"2024-04-26T00:34:54.945950+0000","last_fullsized":"2024-04-26T00:34:54.945950+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":278528,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18719,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21131","reported_seq":28409,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.903313+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:52.903313+0000","last_peered":"2024-04-26T00:34:52.903313+0000","last_clean":"2024-04-26T00:34:52.903313+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:52.903313+0000","last_undegraded":"2024-04-26T00:34:52.903313+0000","last_fullsized":"2024-04-26T00:34:52.903313+0000","mapping_epoch":41,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622616,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18319,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21022","reported_seq":21075,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.957213+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:54.957213+0000","last_peered":"2024-04-26T00:34:54.957213+0000","last_clean":"2024-04-26T00:34:54.957213+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:54.957213+0000","last_undegraded":"2024-04-26T00:34:54.957213+0000","last_fullsized":"2024-04-26T00:34:54.957213+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21022,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21591","reported_seq":21821,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.254086+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:52.254086+0000","last_peered":"2024-04-26T00:34:52.254086+0000","last_clean":"2024-04-26T00:34:52.254086+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:52.254086+0000","last_undegraded":"2024-04-26T00:34:52.254086+0000","last_fullsized":"2024-04-26T00:34:52.254086+0000","mapping_epoch":28,"log_start":"38'11500","ondisk_log_start":"38'11500","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716800,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17992,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21128","reported_seq":21915,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.175453+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:52.175453+0000","last_peered":"2024-04-26T00:34:52.175453+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:52.175453+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602112,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18303,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20787","reported_seq":28159,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.865785+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:52.865785+0000","last_peered":"2024-04-26T00:34:52.865785+0000","last_clean":"2024-04-26T00:34:52.865785+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:52.865785+0000","last_undegraded":"2024-04-26T00:34:52.865785+0000","last_fullsized":"2024-04-26T00:34:52.865785+0000","mapping_epoch":39,"log_start":"38'10700","ondisk_log_start":"38'10700","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":618496,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17989,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21220","reported_seq":21431,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.264481+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:52.264481+0000","last_peered":"2024-04-26T00:34:52.264481+0000","last_clean":"2024-04-26T00:34:52.264481+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:52.264481+0000","last_undegraded":"2024-04-26T00:34:52.264481+0000","last_fullsized":"2024-04-26T00:34:52.264481+0000","mapping_epoch":28,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":761856,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18418,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21001","reported_seq":21568,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:54.995358+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:54.995358+0000","last_peered":"2024-04-26T00:34:54.995358+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:54.995358+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271302,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20627,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21285","reported_seq":22060,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.279524+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:52.279524+0000","last_peered":"2024-04-26T00:34:52.279524+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:52.279524+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18066,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21347","reported_seq":24743,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:54.984097+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:34:54.984097+0000","last_peered":"2024-04-26T00:34:54.984097+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:34:54.984097+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253952,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18347,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1644,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21042","reported_seq":28245,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.934755+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:54.934755+0000","last_peered":"2024-04-26T00:34:54.934755+0000","last_clean":"2024-04-26T00:34:54.934755+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:54.934755+0000","last_undegraded":"2024-04-26T00:34:54.934755+0000","last_fullsized":"2024-04-26T00:34:54.934755+0000","mapping_epoch":37,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319488,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18047,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21121","reported_seq":21849,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.236854+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:52.236854+0000","last_peered":"2024-04-26T00:34:52.236854+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:52.236854+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21018","reported_seq":21659,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.229900+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:52.229900+0000","last_peered":"2024-04-26T00:34:52.229900+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:52.229900+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622592,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17900,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21261","reported_seq":21983,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.249693+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:52.249693+0000","last_peered":"2024-04-26T00:34:52.249693+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:52.249693+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651264,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17957,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3204}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21175","reported_seq":28591,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.917625+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:52.917625+0000","last_peered":"2024-04-26T00:34:52.917625+0000","last_clean":"2024-04-26T00:34:52.917625+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:52.917625+0000","last_undegraded":"2024-04-26T00:34:52.917625+0000","last_fullsized":"2024-04-26T00:34:52.917625+0000","mapping_epoch":43,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602595,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":8623553,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":299069,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17986,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194334720,"data_stored":194272381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503428,"kb_used_data":8060,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93868412,"statfs":{"total":96636764160,"available":96121253888,"internally_reserved":0,"allocated":8253440,"data_stored":7956832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":507244239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280639,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553188,"kb_used_data":66204,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93818652,"statfs":{"total":96636764160,"available":96070299648,"internally_reserved":0,"allocated":67792896,"data_stored":67543716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":498659021},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084412,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484280,"kb_used_data":119396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93887560,"statfs":{"total":96636764160,"available":96140861440,"internally_reserved":0,"allocated":122261504,"data_stored":122008252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":373616336},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66633728,"data_stored":66630115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6901760,"data_stored":6850719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120799232,"data_stored":120791547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:34:57.900 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:34:58.235 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:34:58.235 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:34:58.249 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-04-26T00:34:56.570957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8623553,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":299069,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17986,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540896,"kb_used_data":193660,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281574624,"statfs":{"total":289910292480,"available":288332414976,"internally_reserved":0,"allocated":198307840,"data_stored":197508800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,8,7,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25956352,"num_objects":-6337,"num_object_clones":0,"num_object_copies":-12674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6337,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002576"},"pg_stats":[{"pgid":"2.f","version":"44'21096","reported_seq":21748,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.183152+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:52.183152+0000","last_peered":"2024-04-26T00:34:52.183152+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:52.183152+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":671744,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21096,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21614","reported_seq":21711,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.945950+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:54.945950+0000","last_peered":"2024-04-26T00:34:54.945950+0000","last_clean":"2024-04-26T00:34:54.945950+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:54.945950+0000","last_undegraded":"2024-04-26T00:34:54.945950+0000","last_fullsized":"2024-04-26T00:34:54.945950+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":278528,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18719,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21131","reported_seq":28409,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.903313+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:52.903313+0000","last_peered":"2024-04-26T00:34:52.903313+0000","last_clean":"2024-04-26T00:34:52.903313+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:52.903313+0000","last_undegraded":"2024-04-26T00:34:52.903313+0000","last_fullsized":"2024-04-26T00:34:52.903313+0000","mapping_epoch":41,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622616,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18319,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21022","reported_seq":21075,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.957213+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:54.957213+0000","last_peered":"2024-04-26T00:34:54.957213+0000","last_clean":"2024-04-26T00:34:54.957213+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:54.957213+0000","last_undegraded":"2024-04-26T00:34:54.957213+0000","last_fullsized":"2024-04-26T00:34:54.957213+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21022,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21591","reported_seq":21821,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.254086+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:52.254086+0000","last_peered":"2024-04-26T00:34:52.254086+0000","last_clean":"2024-04-26T00:34:52.254086+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:52.254086+0000","last_undegraded":"2024-04-26T00:34:52.254086+0000","last_fullsized":"2024-04-26T00:34:52.254086+0000","mapping_epoch":28,"log_start":"38'11500","ondisk_log_start":"38'11500","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716800,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17992,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21128","reported_seq":21915,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.175453+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:52.175453+0000","last_peered":"2024-04-26T00:34:52.175453+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:52.175453+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602112,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18303,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20787","reported_seq":28159,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.865785+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:52.865785+0000","last_peered":"2024-04-26T00:34:52.865785+0000","last_clean":"2024-04-26T00:34:52.865785+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:52.865785+0000","last_undegraded":"2024-04-26T00:34:52.865785+0000","last_fullsized":"2024-04-26T00:34:52.865785+0000","mapping_epoch":39,"log_start":"38'10700","ondisk_log_start":"38'10700","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":618496,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17989,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21220","reported_seq":21431,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.264481+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:52.264481+0000","last_peered":"2024-04-26T00:34:52.264481+0000","last_clean":"2024-04-26T00:34:52.264481+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:52.264481+0000","last_undegraded":"2024-04-26T00:34:52.264481+0000","last_fullsized":"2024-04-26T00:34:52.264481+0000","mapping_epoch":28,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":761856,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18418,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21001","reported_seq":21568,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:54.995358+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:54.995358+0000","last_peered":"2024-04-26T00:34:54.995358+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:54.995358+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271302,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20627,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21285","reported_seq":22060,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.279524+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:52.279524+0000","last_peered":"2024-04-26T00:34:52.279524+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:52.279524+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18066,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21347","reported_seq":24743,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:54.984097+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:34:54.984097+0000","last_peered":"2024-04-26T00:34:54.984097+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:34:54.984097+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253952,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18347,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1644,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21042","reported_seq":28245,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.934755+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:54.934755+0000","last_peered":"2024-04-26T00:34:54.934755+0000","last_clean":"2024-04-26T00:34:54.934755+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:54.934755+0000","last_undegraded":"2024-04-26T00:34:54.934755+0000","last_fullsized":"2024-04-26T00:34:54.934755+0000","mapping_epoch":37,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319488,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18047,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21121","reported_seq":21849,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.236854+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:52.236854+0000","last_peered":"2024-04-26T00:34:52.236854+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:52.236854+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21018","reported_seq":21659,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.229900+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:52.229900+0000","last_peered":"2024-04-26T00:34:52.229900+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:52.229900+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622592,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17900,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21261","reported_seq":21983,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.249693+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:52.249693+0000","last_peered":"2024-04-26T00:34:52.249693+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:52.249693+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651264,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17957,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3204}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21175","reported_seq":28591,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.917625+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:52.917625+0000","last_peered":"2024-04-26T00:34:52.917625+0000","last_clean":"2024-04-26T00:34:52.917625+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:52.917625+0000","last_undegraded":"2024-04-26T00:34:52.917625+0000","last_fullsized":"2024-04-26T00:34:52.917625+0000","mapping_epoch":43,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602595,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":8623553,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":299069,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17986,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194334720,"data_stored":194272381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503428,"kb_used_data":8060,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93868412,"statfs":{"total":96636764160,"available":96121253888,"internally_reserved":0,"allocated":8253440,"data_stored":7956832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":507244239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280639,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553188,"kb_used_data":66204,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93818652,"statfs":{"total":96636764160,"available":96070299648,"internally_reserved":0,"allocated":67792896,"data_stored":67543716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":498659021},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084412,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484280,"kb_used_data":119396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93887560,"statfs":{"total":96636764160,"available":96140861440,"internally_reserved":0,"allocated":122261504,"data_stored":122008252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":373616336},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66633728,"data_stored":66630115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6901760,"data_stored":6850719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120799232,"data_stored":120791547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:34:58.251 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:34:58.554 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:34:58.555 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:34:58.567 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":359,"stamp":"2024-04-26T00:34:56.570957+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":8623553,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":299069,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17986,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1540896,"kb_used_data":193660,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281574624,"statfs":{"total":289910292480,"available":288332414976,"internally_reserved":0,"allocated":198307840,"data_stored":197508800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,0,8,7,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25956352,"num_objects":-6337,"num_object_clones":0,"num_object_copies":-12674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6337,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002576"},"pg_stats":[{"pgid":"2.f","version":"44'21096","reported_seq":21748,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.183152+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:52.183152+0000","last_peered":"2024-04-26T00:34:52.183152+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:52.183152+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":671744,"num_objects":164,"num_object_clones":0,"num_object_copies":328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21096,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3180}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21614","reported_seq":21711,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.945950+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:54.945950+0000","last_peered":"2024-04-26T00:34:54.945950+0000","last_clean":"2024-04-26T00:34:54.945950+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:54.945950+0000","last_undegraded":"2024-04-26T00:34:54.945950+0000","last_fullsized":"2024-04-26T00:34:54.945950+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":278528,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18719,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21131","reported_seq":28409,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.903313+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:52.903313+0000","last_peered":"2024-04-26T00:34:52.903313+0000","last_clean":"2024-04-26T00:34:52.903313+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:52.903313+0000","last_undegraded":"2024-04-26T00:34:52.903313+0000","last_fullsized":"2024-04-26T00:34:52.903313+0000","mapping_epoch":41,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622616,"num_objects":153,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18319,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21022","reported_seq":21075,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.957213+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:54.957213+0000","last_peered":"2024-04-26T00:34:54.957213+0000","last_clean":"2024-04-26T00:34:54.957213+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:54.957213+0000","last_undegraded":"2024-04-26T00:34:54.957213+0000","last_fullsized":"2024-04-26T00:34:54.957213+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21022,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21591","reported_seq":21821,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.254086+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:52.254086+0000","last_peered":"2024-04-26T00:34:52.254086+0000","last_clean":"2024-04-26T00:34:52.254086+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:52.254086+0000","last_undegraded":"2024-04-26T00:34:52.254086+0000","last_fullsized":"2024-04-26T00:34:52.254086+0000","mapping_epoch":28,"log_start":"38'11500","ondisk_log_start":"38'11500","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716800,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17992,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21128","reported_seq":21915,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.175453+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:52.175453+0000","last_peered":"2024-04-26T00:34:52.175453+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:52.175453+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602112,"num_objects":147,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18303,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3099}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20787","reported_seq":28159,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.865785+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:52.865785+0000","last_peered":"2024-04-26T00:34:52.865785+0000","last_clean":"2024-04-26T00:34:52.865785+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:52.865785+0000","last_undegraded":"2024-04-26T00:34:52.865785+0000","last_fullsized":"2024-04-26T00:34:52.865785+0000","mapping_epoch":39,"log_start":"38'10700","ondisk_log_start":"38'10700","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":618496,"num_objects":151,"num_object_clones":0,"num_object_copies":302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17989,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21220","reported_seq":21431,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.264481+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:52.264481+0000","last_peered":"2024-04-26T00:34:52.264481+0000","last_clean":"2024-04-26T00:34:52.264481+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:52.264481+0000","last_undegraded":"2024-04-26T00:34:52.264481+0000","last_fullsized":"2024-04-26T00:34:52.264481+0000","mapping_epoch":28,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":761856,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18418,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21001","reported_seq":21568,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:54.995358+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:54.995358+0000","last_peered":"2024-04-26T00:34:54.995358+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:54.995358+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271302,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20627,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21285","reported_seq":22060,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.279524+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:52.279524+0000","last_peered":"2024-04-26T00:34:52.279524+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:52.279524+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":569344,"num_objects":139,"num_object_clones":0,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18066,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3161}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21347","reported_seq":24743,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:54.984097+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:34:54.984097+0000","last_peered":"2024-04-26T00:34:54.984097+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:34:54.984097+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253952,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18347,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1644,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21042","reported_seq":28245,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.934755+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:54.934755+0000","last_peered":"2024-04-26T00:34:54.934755+0000","last_clean":"2024-04-26T00:34:54.934755+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:54.934755+0000","last_undegraded":"2024-04-26T00:34:54.934755+0000","last_fullsized":"2024-04-26T00:34:54.934755+0000","mapping_epoch":37,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319488,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18047,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21121","reported_seq":21849,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.236854+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:52.236854+0000","last_peered":"2024-04-26T00:34:52.236854+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:52.236854+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21018","reported_seq":21659,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.229900+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:52.229900+0000","last_peered":"2024-04-26T00:34:52.229900+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:52.229900+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622592,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17900,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21261","reported_seq":21983,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:52.249693+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:52.249693+0000","last_peered":"2024-04-26T00:34:52.249693+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:52.249693+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":651264,"num_objects":159,"num_object_clones":0,"num_object_copies":318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17957,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3204}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21175","reported_seq":28591,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:52.917625+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:52.917625+0000","last_peered":"2024-04-26T00:34:52.917625+0000","last_clean":"2024-04-26T00:34:52.917625+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:52.917625+0000","last_undegraded":"2024-04-26T00:34:52.917625+0000","last_fullsized":"2024-04-26T00:34:52.917625+0000","mapping_epoch":43,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":602595,"num_objects":148,"num_object_clones":0,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":8623553,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":299069,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17986,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194334720,"data_stored":194272381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160739,"ondisk_log_size":160739,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":503428,"kb_used_data":8060,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93868412,"statfs":{"total":96636764160,"available":96121253888,"internally_reserved":0,"allocated":8253440,"data_stored":7956832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":507244239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280639,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553188,"kb_used_data":66204,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93818652,"statfs":{"total":96636764160,"available":96070299648,"internally_reserved":0,"allocated":67792896,"data_stored":67543716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":498659021},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,0,3,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084412,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484280,"kb_used_data":119396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93887560,"statfs":{"total":96636764160,"available":96140861440,"internally_reserved":0,"allocated":122261504,"data_stored":122008252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":373616336},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":66633728,"data_stored":66630115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":6901760,"data_stored":6850719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120799232,"data_stored":120791547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:34:58.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-04-26T00:34:58.568 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'21096", 'reported_seq': 21748, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:52.183152+0000', 'last_change': '2024-04-26T00:26:01.837535+0000', 'last_active': '2024-04-26T00:34:52.183152+0000', 'last_peered': '2024-04-26T00:34:52.183152+0000', 'last_clean': '2024-04-26T00:25:59.337291+0000', 'last_became_active': '2024-04-26T00:26:01.832792+0000', 'last_became_peered': '2024-04-26T00:26:01.832792+0000', 'last_unstale': '2024-04-26T00:34:52.183152+0000', 'last_undegraded': '2024-04-26T00:26:01.450456+0000', 'last_fullsized': '2024-04-26T00:26:01.360942+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 0, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 671744, 'num_objects': 164, 'num_object_clones': 0, 'num_object_copies': 328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3344, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 164, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21096, 'num_write_kb': 42520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 287, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3180}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21128", 'reported_seq': 21915, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:52.175453+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:34:52.175453+0000', 'last_peered': '2024-04-26T00:34:52.175453+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:34:52.175453+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 602112, 'num_objects': 147, 'num_object_clones': 0, 'num_object_copies': 294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18303, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3099}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21001", 'reported_seq': 21568, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:54.995358+0000', 'last_change': '2024-04-26T00:26:01.842290+0000', 'last_active': '2024-04-26T00:34:54.995358+0000', 'last_peered': '2024-04-26T00:34:54.995358+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:34:54.995358+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 271302, 'num_objects': 68, 'num_object_clones': 0, 'num_object_copies': 136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 68, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20627, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3215}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21285", 'reported_seq': 22060, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:52.279524+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:34:52.279524+0000', 'last_peered': '2024-04-26T00:34:52.279524+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:34:52.279524+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 569344, 'num_objects': 139, 'num_object_clones': 0, 'num_object_copies': 278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 139, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18066, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3161}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'21347", 'reported_seq': 24743, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:54.984097+0000', 'last_change': '2024-04-26T00:33:54.556624+0000', 'last_active': '2024-04-26T00:34:54.984097+0000', 'last_peered': '2024-04-26T00:34:54.984097+0000', 'last_clean': '2024-04-26T00:25:59.337465+0000', 'last_became_active': '2024-04-26T00:26:01.806140+0000', 'last_became_peered': '2024-04-26T00:26:01.806140+0000', 'last_unstale': '2024-04-26T00:34:54.984097+0000', 'last_undegraded': '2024-04-26T00:26:01.404389+0000', 'last_fullsized': '2024-04-26T00:26:01.366785+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3022", 'last_scrub_stamp': '2024-04-26T00:25:47.736821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:47.736821+0000', 'objects_scrubbed': 2995, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 758, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 253952, 'num_objects': 62, 'num_object_clones': 0, 'num_object_copies': 124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2039, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 62, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18347, 'num_write_kb': 30828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1644, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1977}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:34:58.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21121", 'reported_seq': 21849, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:52.236854+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:34:52.236854+0000', 'last_peered': '2024-04-26T00:34:52.236854+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:34:52.236854+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 749568, 'num_objects': 183, 'num_object_clones': 0, 'num_object_copies': 366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18042, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3152}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:34:58.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:34:58.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21018", 'reported_seq': 21659, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:52.229900+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:34:52.229900+0000', 'last_peered': '2024-04-26T00:34:52.229900+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:34:52.229900+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 622592, 'num_objects': 152, 'num_object_clones': 0, 'num_object_copies': 304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 152, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17900, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3151}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:34:58.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:34:58.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21261", 'reported_seq': 21983, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:52.249693+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:34:52.249693+0000', 'last_peered': '2024-04-26T00:34:52.249693+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:34:52.249693+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 651264, 'num_objects': 159, 'num_object_clones': 0, 'num_object_copies': 318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17957, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3204}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:34:58.570 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:34:58.972 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:34:58.973 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25225/706 objects degraded (3572.946%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":644,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":353,"data_bytes":1435049,"bytes_used":1553162240,"bytes_avail":288357130240,"bytes_total":289910292480,"degraded_objects":25225,"degraded_total":706,"degraded_ratio":35.729461756373937,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":600},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-04-26T00:34:52.570180+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:34:58.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:34:58.987 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:34:59.291 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:34:59.291 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:34:59.303 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-04-26T00:34:58.571392+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1435049,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":300825,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17986,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160595,"ondisk_log_size":160595,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516760,"kb_used_data":177836,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281598760,"statfs":{"total":289910292480,"available":288357130240,"internally_reserved":0,"allocated":182104064,"data_stored":181309096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1370999916},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,10,6,8,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-29519896,"num_objects":-7208,"num_object_clones":0,"num_object_copies":-14416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-7208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7208,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002589"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21614","reported_seq":21711,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.945950+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:54.945950+0000","last_peered":"2024-04-26T00:34:54.945950+0000","last_clean":"2024-04-26T00:34:54.945950+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:54.945950+0000","last_undegraded":"2024-04-26T00:34:54.945950+0000","last_fullsized":"2024-04-26T00:34:54.945950+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":278528,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18719,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28562,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.078448+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:57.078448+0000","last_peered":"2024-04-26T00:34:57.078448+0000","last_clean":"2024-04-26T00:34:57.078448+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:57.078448+0000","last_undegraded":"2024-04-26T00:34:57.078448+0000","last_fullsized":"2024-04-26T00:34:57.078448+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21022","reported_seq":21075,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.957213+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:54.957213+0000","last_peered":"2024-04-26T00:34:54.957213+0000","last_clean":"2024-04-26T00:34:54.957213+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:54.957213+0000","last_undegraded":"2024-04-26T00:34:54.957213+0000","last_fullsized":"2024-04-26T00:34:54.957213+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":311296,"num_objects":76,"num_object_clones":0,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21022,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28310,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058776+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:57.058776+0000","last_peered":"2024-04-26T00:34:57.058776+0000","last_clean":"2024-04-26T00:34:57.058776+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:57.058776+0000","last_undegraded":"2024-04-26T00:34:57.058776+0000","last_fullsized":"2024-04-26T00:34:57.058776+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21001","reported_seq":21568,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:54.995358+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:54.995358+0000","last_peered":"2024-04-26T00:34:54.995358+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:54.995358+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":271302,"num_objects":68,"num_object_clones":0,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20627,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21347","reported_seq":24743,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:54.984097+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:34:54.984097+0000","last_peered":"2024-04-26T00:34:54.984097+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:34:54.984097+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":253952,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2039,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18347,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1644,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1977}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21042","reported_seq":28245,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:54.934755+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:54.934755+0000","last_peered":"2024-04-26T00:34:54.934755+0000","last_clean":"2024-04-26T00:34:54.934755+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:54.934755+0000","last_undegraded":"2024-04-26T00:34:54.934755+0000","last_fullsized":"2024-04-26T00:34:54.934755+0000","mapping_epoch":37,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319488,"num_objects":78,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18047,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28738,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:34:57.041276+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:34:57.041276+0000","last_fullsized":"2024-04-26T00:34:57.041276+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1435049,"num_objects":353,"num_object_clones":0,"num_object_copies":706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":353,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":300825,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17986,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179314688,"data_stored":179256421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160595,"ondisk_log_size":160595,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483300,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888540,"statfs":{"total":96636764160,"available":96141864960,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280640,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":549180,"kb_used_data":58028,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93822660,"statfs":{"total":96636764160,"available":96074403840,"internally_reserved":0,"allocated":59420672,"data_stored":59171492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084412,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484280,"kb_used_data":119396,"kb_used_omap":4,"kb_used_meta":364859,"kb_avail":93887560,"statfs":{"total":96636764160,"available":96140861440,"internally_reserved":0,"allocated":122261504,"data_stored":122008252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":373616336},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120799232,"data_stored":120791547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:02.305 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:02.603 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:02.603 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:02.615 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-26T00:35:02.572053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18469,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516948,"kb_used_data":173884,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598572,"statfs":{"total":289910292480,"available":288356937728,"internally_reserved":0,"allocated":178057216,"data_stored":177266320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19734552,"num_objects":-4819,"num_object_clones":0,"num_object_copies":-9638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002358"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21779,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.075886+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:57.075886+0000","last_peered":"2024-04-26T00:34:57.075886+0000","last_clean":"2024-04-26T00:34:57.075886+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:57.075886+0000","last_undegraded":"2024-04-26T00:34:57.075886+0000","last_fullsized":"2024-04-26T00:34:57.075886+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28562,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.078448+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:57.078448+0000","last_peered":"2024-04-26T00:34:57.078448+0000","last_clean":"2024-04-26T00:34:57.078448+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:57.078448+0000","last_undegraded":"2024-04-26T00:34:57.078448+0000","last_fullsized":"2024-04-26T00:34:57.078448+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.084886+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:57.084886+0000","last_peered":"2024-04-26T00:34:57.084886+0000","last_clean":"2024-04-26T00:34:57.084886+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:57.084886+0000","last_undegraded":"2024-04-26T00:34:57.084886+0000","last_fullsized":"2024-04-26T00:34:57.084886+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28310,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058776+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:57.058776+0000","last_peered":"2024-04-26T00:34:57.058776+0000","last_clean":"2024-04-26T00:34:57.058776+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:57.058776+0000","last_undegraded":"2024-04-26T00:34:57.058776+0000","last_fullsized":"2024-04-26T00:34:57.058776+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21634,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.032435+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:57.032435+0000","last_peered":"2024-04-26T00:34:57.032435+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:57.032435+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":25771,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:00.004333+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:35:00.004333+0000","last_peered":"2024-04-26T00:35:00.004333+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:35:00.004333+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1556}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28323,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.022287+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:57.022287+0000","last_peered":"2024-04-26T00:34:57.022287+0000","last_clean":"2024-04-26T00:34:57.022287+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:57.022287+0000","last_undegraded":"2024-04-26T00:34:57.022287+0000","last_fullsized":"2024-04-26T00:34:57.022287+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28738,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:34:57.041276+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:34:57.041276+0000","last_fullsized":"2024-04-26T00:34:57.041276+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18469,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483300,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888540,"statfs":{"total":96636764160,"available":96141864960,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280641,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084413,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485052,"kb_used_data":116020,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886788,"statfs":{"total":96636764160,"available":96140070912,"internally_reserved":0,"allocated":118804480,"data_stored":118555300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:02.616 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:02.931 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:02.931 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:02.944 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-26T00:35:02.572053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18469,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516948,"kb_used_data":173884,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598572,"statfs":{"total":289910292480,"available":288356937728,"internally_reserved":0,"allocated":178057216,"data_stored":177266320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19734552,"num_objects":-4819,"num_object_clones":0,"num_object_copies":-9638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002358"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21779,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.075886+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:57.075886+0000","last_peered":"2024-04-26T00:34:57.075886+0000","last_clean":"2024-04-26T00:34:57.075886+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:57.075886+0000","last_undegraded":"2024-04-26T00:34:57.075886+0000","last_fullsized":"2024-04-26T00:34:57.075886+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28562,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.078448+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:57.078448+0000","last_peered":"2024-04-26T00:34:57.078448+0000","last_clean":"2024-04-26T00:34:57.078448+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:57.078448+0000","last_undegraded":"2024-04-26T00:34:57.078448+0000","last_fullsized":"2024-04-26T00:34:57.078448+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.084886+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:57.084886+0000","last_peered":"2024-04-26T00:34:57.084886+0000","last_clean":"2024-04-26T00:34:57.084886+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:57.084886+0000","last_undegraded":"2024-04-26T00:34:57.084886+0000","last_fullsized":"2024-04-26T00:34:57.084886+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28310,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058776+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:57.058776+0000","last_peered":"2024-04-26T00:34:57.058776+0000","last_clean":"2024-04-26T00:34:57.058776+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:57.058776+0000","last_undegraded":"2024-04-26T00:34:57.058776+0000","last_fullsized":"2024-04-26T00:34:57.058776+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21634,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.032435+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:57.032435+0000","last_peered":"2024-04-26T00:34:57.032435+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:57.032435+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":25771,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:00.004333+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:35:00.004333+0000","last_peered":"2024-04-26T00:35:00.004333+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:35:00.004333+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1556}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28323,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.022287+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:57.022287+0000","last_peered":"2024-04-26T00:34:57.022287+0000","last_clean":"2024-04-26T00:34:57.022287+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:57.022287+0000","last_undegraded":"2024-04-26T00:34:57.022287+0000","last_fullsized":"2024-04-26T00:34:57.022287+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28738,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:34:57.041276+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:34:57.041276+0000","last_fullsized":"2024-04-26T00:34:57.041276+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18469,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483300,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888540,"statfs":{"total":96636764160,"available":96141864960,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280641,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084413,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485052,"kb_used_data":116020,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886788,"statfs":{"total":96636764160,"available":96140070912,"internally_reserved":0,"allocated":118804480,"data_stored":118555300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:02.945 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-04-26T00:35:02.945 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'21260", 'reported_seq': 21913, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.081144+0000', 'last_change': '2024-04-26T00:26:01.837535+0000', 'last_active': '2024-04-26T00:34:57.081144+0000', 'last_peered': '2024-04-26T00:34:57.081144+0000', 'last_clean': '2024-04-26T00:25:59.337291+0000', 'last_became_active': '2024-04-26T00:26:01.832792+0000', 'last_became_peered': '2024-04-26T00:26:01.832792+0000', 'last_unstale': '2024-04-26T00:34:57.081144+0000', 'last_undegraded': '2024-04-26T00:26:01.450456+0000', 'last_fullsized': '2024-04-26T00:26:01.360942+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3344, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21260, 'num_write_kb': 42520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 287, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3344}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22062, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.041276+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:34:57.041276+0000', 'last_peered': '2024-04-26T00:34:57.041276+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:34:57.041276+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 21634, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.032435+0000', 'last_change': '2024-04-26T00:26:01.842290+0000', 'last_active': '2024-04-26T00:34:57.032435+0000', 'last_peered': '2024-04-26T00:34:57.032435+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:34:57.032435+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22199, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:56.986322+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:34:56.986322+0000', 'last_peered': '2024-04-26T00:34:56.986322+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:34:56.986322+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'21409", 'reported_seq': 25771, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:00.004333+0000', 'last_change': '2024-04-26T00:33:54.556624+0000', 'last_active': '2024-04-26T00:35:00.004333+0000', 'last_peered': '2024-04-26T00:35:00.004333+0000', 'last_clean': '2024-04-26T00:25:59.337465+0000', 'last_became_active': '2024-04-26T00:26:01.806140+0000', 'last_became_peered': '2024-04-26T00:26:01.806140+0000', 'last_unstale': '2024-04-26T00:35:00.004333+0000', 'last_undegraded': '2024-04-26T00:26:01.404389+0000', 'last_fullsized': '2024-04-26T00:26:01.366785+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3022", 'last_scrub_stamp': '2024-04-26T00:25:47.736821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:47.736821+0000', 'objects_scrubbed': 2995, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 758, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1556, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18409, 'num_write_kb': 30828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2127, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1556}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:02.946 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22034, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.009637+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:34:57.009637+0000', 'last_peered': '2024-04-26T00:34:57.009637+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:34:57.009637+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:02.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:02.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21811, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.022726+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:34:57.022726+0000', 'last_peered': '2024-04-26T00:34:57.022726+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:34:57.022726+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:02.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:02.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22142, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.069595+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:34:57.069595+0000', 'last_peered': '2024-04-26T00:34:57.069595+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:34:57.069595+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:02.947 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:03.320 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:03.321 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24742/6 objects degraded (412366.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":649,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1553354752,"bytes_avail":288356937728,"bytes_total":289910292480,"degraded_objects":24742,"degraded_total":6,"degraded_ratio":4123.666666666667,"recovering_objects_per_sec":49,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":590,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":401},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-04-26T00:34:52.570180+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:35:03.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:03.333 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:03.631 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:03.631 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:03.643 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-04-26T00:35:02.572053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18469,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516948,"kb_used_data":173884,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598572,"statfs":{"total":289910292480,"available":288356937728,"internally_reserved":0,"allocated":178057216,"data_stored":177266320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19734552,"num_objects":-4819,"num_object_clones":0,"num_object_copies":-9638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-590,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4819,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4819,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002358"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21779,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.075886+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:57.075886+0000","last_peered":"2024-04-26T00:34:57.075886+0000","last_clean":"2024-04-26T00:34:57.075886+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:57.075886+0000","last_undegraded":"2024-04-26T00:34:57.075886+0000","last_fullsized":"2024-04-26T00:34:57.075886+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28562,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.078448+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:57.078448+0000","last_peered":"2024-04-26T00:34:57.078448+0000","last_clean":"2024-04-26T00:34:57.078448+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:57.078448+0000","last_undegraded":"2024-04-26T00:34:57.078448+0000","last_fullsized":"2024-04-26T00:34:57.078448+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.084886+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:57.084886+0000","last_peered":"2024-04-26T00:34:57.084886+0000","last_clean":"2024-04-26T00:34:57.084886+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:57.084886+0000","last_undegraded":"2024-04-26T00:34:57.084886+0000","last_fullsized":"2024-04-26T00:34:57.084886+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28310,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058776+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:57.058776+0000","last_peered":"2024-04-26T00:34:57.058776+0000","last_clean":"2024-04-26T00:34:57.058776+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:57.058776+0000","last_undegraded":"2024-04-26T00:34:57.058776+0000","last_fullsized":"2024-04-26T00:34:57.058776+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21634,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.032435+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:57.032435+0000","last_peered":"2024-04-26T00:34:57.032435+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:57.032435+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":25771,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:00.004333+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:35:00.004333+0000","last_peered":"2024-04-26T00:35:00.004333+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:35:00.004333+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1556}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28323,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.022287+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:57.022287+0000","last_peered":"2024-04-26T00:34:57.022287+0000","last_clean":"2024-04-26T00:34:57.022287+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:57.022287+0000","last_undegraded":"2024-04-26T00:34:57.022287+0000","last_fullsized":"2024-04-26T00:34:57.022287+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28738,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:34:57.041276+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:34:57.041276+0000","last_fullsized":"2024-04-26T00:34:57.041276+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24742,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18469,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483300,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888540,"statfs":{"total":96636764160,"available":96141864960,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,3,0,1,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280641,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084413,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485052,"kb_used_data":116020,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886788,"statfs":{"total":96636764160,"available":96140070912,"internally_reserved":0,"allocated":118804480,"data_stored":118555300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:06.646 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:06.964 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:06.965 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:06.977 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-04-26T00:35:06.572700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19450,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525268,"kb_used_data":173884,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590252,"statfs":{"total":289910292480,"available":288348418048,"internally_reserved":0,"allocated":178057216,"data_stored":177266320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12099608,"num_objects":-2955,"num_object_clones":0,"num_object_copies":-5910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1571,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002162"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21779,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.075886+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:57.075886+0000","last_peered":"2024-04-26T00:34:57.075886+0000","last_clean":"2024-04-26T00:34:57.075886+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:57.075886+0000","last_undegraded":"2024-04-26T00:34:57.075886+0000","last_fullsized":"2024-04-26T00:34:57.075886+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28562,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.078448+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:57.078448+0000","last_peered":"2024-04-26T00:34:57.078448+0000","last_clean":"2024-04-26T00:34:57.078448+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:57.078448+0000","last_undegraded":"2024-04-26T00:34:57.078448+0000","last_fullsized":"2024-04-26T00:34:57.078448+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.084886+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:57.084886+0000","last_peered":"2024-04-26T00:34:57.084886+0000","last_clean":"2024-04-26T00:34:57.084886+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:57.084886+0000","last_undegraded":"2024-04-26T00:34:57.084886+0000","last_fullsized":"2024-04-26T00:34:57.084886+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28310,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058776+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:57.058776+0000","last_peered":"2024-04-26T00:34:57.058776+0000","last_clean":"2024-04-26T00:34:57.058776+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:57.058776+0000","last_undegraded":"2024-04-26T00:34:57.058776+0000","last_fullsized":"2024-04-26T00:34:57.058776+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21634,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.032435+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:57.032435+0000","last_peered":"2024-04-26T00:34:57.032435+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:57.032435+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":27733,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:05.007082+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:35:05.007082+0000","last_peered":"2024-04-26T00:35:05.007082+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:35:05.007082+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-575}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28323,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.022287+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:57.022287+0000","last_peered":"2024-04-26T00:34:57.022287+0000","last_clean":"2024-04-26T00:34:57.022287+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:57.022287+0000","last_undegraded":"2024-04-26T00:34:57.022287+0000","last_fullsized":"2024-04-26T00:34:57.022287+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28738,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:34:57.041276+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:34:57.041276+0000","last_fullsized":"2024-04-26T00:34:57.041276+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19450,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491620,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880220,"statfs":{"total":96636764160,"available":96133345280,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280641,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084414,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485052,"kb_used_data":116020,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886788,"statfs":{"total":96636764160,"available":96140070912,"internally_reserved":0,"allocated":118804480,"data_stored":118555300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:06.978 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:07.273 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:07.274 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:07.286 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-04-26T00:35:06.572700+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19450,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525268,"kb_used_data":173884,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590252,"statfs":{"total":289910292480,"available":288348418048,"internally_reserved":0,"allocated":178057216,"data_stored":177266320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12099608,"num_objects":-2955,"num_object_clones":0,"num_object_copies":-5910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1571,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2955,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1571,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002162"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21779,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.075886+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:57.075886+0000","last_peered":"2024-04-26T00:34:57.075886+0000","last_clean":"2024-04-26T00:34:57.075886+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:57.075886+0000","last_undegraded":"2024-04-26T00:34:57.075886+0000","last_fullsized":"2024-04-26T00:34:57.075886+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28562,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.078448+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:57.078448+0000","last_peered":"2024-04-26T00:34:57.078448+0000","last_clean":"2024-04-26T00:34:57.078448+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:57.078448+0000","last_undegraded":"2024-04-26T00:34:57.078448+0000","last_fullsized":"2024-04-26T00:34:57.078448+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.084886+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:57.084886+0000","last_peered":"2024-04-26T00:34:57.084886+0000","last_clean":"2024-04-26T00:34:57.084886+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:57.084886+0000","last_undegraded":"2024-04-26T00:34:57.084886+0000","last_fullsized":"2024-04-26T00:34:57.084886+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28310,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058776+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:57.058776+0000","last_peered":"2024-04-26T00:34:57.058776+0000","last_clean":"2024-04-26T00:34:57.058776+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:57.058776+0000","last_undegraded":"2024-04-26T00:34:57.058776+0000","last_fullsized":"2024-04-26T00:34:57.058776+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21634,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.032435+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:57.032435+0000","last_peered":"2024-04-26T00:34:57.032435+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:57.032435+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":27733,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:05.007082+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:35:05.007082+0000","last_peered":"2024-04-26T00:35:05.007082+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:35:05.007082+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-575}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28323,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.022287+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:57.022287+0000","last_peered":"2024-04-26T00:34:57.022287+0000","last_clean":"2024-04-26T00:34:57.022287+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:57.022287+0000","last_undegraded":"2024-04-26T00:34:57.022287+0000","last_fullsized":"2024-04-26T00:34:57.022287+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28738,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:34:57.041276+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:34:57.041276+0000","last_fullsized":"2024-04-26T00:34:57.041276+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19450,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491620,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880220,"statfs":{"total":96636764160,"available":96133345280,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280641,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084414,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485052,"kb_used_data":116020,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886788,"statfs":{"total":96636764160,"available":96140070912,"internally_reserved":0,"allocated":118804480,"data_stored":118555300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:07.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-04-26T00:35:07.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'21260", 'reported_seq': 21913, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.081144+0000', 'last_change': '2024-04-26T00:26:01.837535+0000', 'last_active': '2024-04-26T00:34:57.081144+0000', 'last_peered': '2024-04-26T00:34:57.081144+0000', 'last_clean': '2024-04-26T00:25:59.337291+0000', 'last_became_active': '2024-04-26T00:26:01.832792+0000', 'last_became_peered': '2024-04-26T00:26:01.832792+0000', 'last_unstale': '2024-04-26T00:34:57.081144+0000', 'last_undegraded': '2024-04-26T00:26:01.450456+0000', 'last_fullsized': '2024-04-26T00:26:01.360942+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3344, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21260, 'num_write_kb': 42520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 287, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3344}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:07.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:07.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22062, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.041276+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:34:57.041276+0000', 'last_peered': '2024-04-26T00:34:57.041276+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:34:57.041276+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:07.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:07.287 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 21634, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.032435+0000', 'last_change': '2024-04-26T00:26:01.842290+0000', 'last_active': '2024-04-26T00:34:57.032435+0000', 'last_peered': '2024-04-26T00:34:57.032435+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:34:57.032435+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22199, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:56.986322+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:34:56.986322+0000', 'last_peered': '2024-04-26T00:34:56.986322+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:34:56.986322+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "44'21409", 'reported_seq': 27733, 'reported_epoch': 44, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:05.007082+0000', 'last_change': '2024-04-26T00:33:54.556624+0000', 'last_active': '2024-04-26T00:35:05.007082+0000', 'last_peered': '2024-04-26T00:35:05.007082+0000', 'last_clean': '2024-04-26T00:25:59.337465+0000', 'last_became_active': '2024-04-26T00:26:01.806140+0000', 'last_became_peered': '2024-04-26T00:26:01.806140+0000', 'last_unstale': '2024-04-26T00:35:05.007082+0000', 'last_undegraded': '2024-04-26T00:26:01.404389+0000', 'last_fullsized': '2024-04-26T00:26:01.366785+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3022", 'last_scrub_stamp': '2024-04-26T00:25:47.736821+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:47.736821+0000', 'objects_scrubbed': 2995, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 758, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 575, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18409, 'num_write_kb': 30828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3108, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -575}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22034, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.009637+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:34:57.009637+0000', 'last_peered': '2024-04-26T00:34:57.009637+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:34:57.009637+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21811, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.022726+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:34:57.022726+0000', 'last_peered': '2024-04-26T00:34:57.022726+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:34:57.022726+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:07.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22142, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.069595+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:34:57.069595+0000', 'last_peered': '2024-04-26T00:34:57.069595+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:34:57.069595+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:07.289 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:07.664 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:07.664 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23761/6 objects degraded (396016.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":653,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":44,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1561874432,"bytes_avail":288348418048,"bytes_total":289910292480,"degraded_objects":23761,"degraded_total":6,"degraded_ratio":3960.1666666666661,"recovering_objects_per_sec":130,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1571,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":246},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-04-26T00:34:52.570180+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:35:07.678 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:07.678 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:08.037 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:08.038 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:08.051 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-04-26T00:35:07.908599+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19450,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525268,"kb_used_data":173884,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590252,"statfs":{"total":289910292480,"available":288348418048,"internally_reserved":0,"allocated":178057216,"data_stored":177266320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-8622104,"num_objects":-2106,"num_object_clones":0,"num_object_copies":-4212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2106,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.337642"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21779,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.075886+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:34:57.075886+0000","last_peered":"2024-04-26T00:34:57.075886+0000","last_clean":"2024-04-26T00:34:57.075886+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:34:57.075886+0000","last_undegraded":"2024-04-26T00:34:57.075886+0000","last_fullsized":"2024-04-26T00:34:57.075886+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28562,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.078448+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:34:57.078448+0000","last_peered":"2024-04-26T00:34:57.078448+0000","last_clean":"2024-04-26T00:34:57.078448+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:34:57.078448+0000","last_undegraded":"2024-04-26T00:34:57.078448+0000","last_fullsized":"2024-04-26T00:34:57.078448+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21151,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.084886+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:34:57.084886+0000","last_peered":"2024-04-26T00:34:57.084886+0000","last_clean":"2024-04-26T00:34:57.084886+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:34:57.084886+0000","last_undegraded":"2024-04-26T00:34:57.084886+0000","last_fullsized":"2024-04-26T00:34:57.084886+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28310,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058776+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:34:57.058776+0000","last_peered":"2024-04-26T00:34:57.058776+0000","last_clean":"2024-04-26T00:34:57.058776+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:34:57.058776+0000","last_undegraded":"2024-04-26T00:34:57.058776+0000","last_fullsized":"2024-04-26T00:34:57.058776+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411214+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:33:56.411214+0000","last_peered":"2024-04-26T00:33:56.411214+0000","last_clean":"2024-04-26T00:33:56.411214+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:33:56.411214+0000","last_undegraded":"2024-04-26T00:33:56.411214+0000","last_fullsized":"2024-04-26T00:33:56.411214+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21634,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.032435+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:34:57.032435+0000","last_peered":"2024-04-26T00:34:57.032435+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:34:57.032435+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421231+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:33:56.421231+0000","last_peered":"2024-04-26T00:33:56.421231+0000","last_clean":"2024-04-26T00:33:56.421231+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:33:56.421231+0000","last_undegraded":"2024-04-26T00:33:56.421231+0000","last_fullsized":"2024-04-26T00:33:56.421231+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":27733,"reported_epoch":44,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:05.007082+0000","last_change":"2024-04-26T00:33:54.556624+0000","last_active":"2024-04-26T00:35:05.007082+0000","last_peered":"2024-04-26T00:35:05.007082+0000","last_clean":"2024-04-26T00:25:59.337465+0000","last_became_active":"2024-04-26T00:26:01.806140+0000","last_became_peered":"2024-04-26T00:26:01.806140+0000","last_unstale":"2024-04-26T00:35:05.007082+0000","last_undegraded":"2024-04-26T00:26:01.404389+0000","last_fullsized":"2024-04-26T00:26:01.366785+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":575,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3108,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-575}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411321+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:33:56.411321+0000","last_peered":"2024-04-26T00:33:56.411321+0000","last_clean":"2024-04-26T00:33:56.411321+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:33:56.411321+0000","last_undegraded":"2024-04-26T00:33:56.411321+0000","last_fullsized":"2024-04-26T00:33:56.411321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28323,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.022287+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:34:57.022287+0000","last_peered":"2024-04-26T00:34:57.022287+0000","last_clean":"2024-04-26T00:34:57.022287+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:34:57.022287+0000","last_undegraded":"2024-04-26T00:34:57.022287+0000","last_fullsized":"2024-04-26T00:34:57.022287+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411339+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:33:56.411339+0000","last_peered":"2024-04-26T00:33:56.411339+0000","last_clean":"2024-04-26T00:33:56.411339+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:33:56.411339+0000","last_undegraded":"2024-04-26T00:33:56.411339+0000","last_fullsized":"2024-04-26T00:33:56.411339+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":54,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.421795+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:33:56.421795+0000","last_peered":"2024-04-26T00:33:56.421795+0000","last_clean":"2024-04-26T00:33:56.421795+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:33:56.421795+0000","last_undegraded":"2024-04-26T00:33:56.421795+0000","last_fullsized":"2024-04-26T00:33:56.421795+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":71,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411345+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:33:56.411345+0000","last_peered":"2024-04-26T00:33:56.411345+0000","last_clean":"2024-04-26T00:33:56.411345+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:33:56.411345+0000","last_undegraded":"2024-04-26T00:33:56.411345+0000","last_fullsized":"2024-04-26T00:33:56.411345+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.411513+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:33:56.411513+0000","last_peered":"2024-04-26T00:33:56.411513+0000","last_clean":"2024-04-26T00:33:56.411513+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:33:56.411513+0000","last_undegraded":"2024-04-26T00:33:56.411513+0000","last_fullsized":"2024-04-26T00:33:56.411513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28738,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:34:57.041276+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:34:57.041276+0000","last_fullsized":"2024-04-26T00:34:57.041276+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19450,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491620,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880220,"statfs":{"total":96636764160,"available":96133345280,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280642,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084414,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485052,"kb_used_data":116020,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886788,"statfs":{"total":96636764160,"available":96140070912,"internally_reserved":0,"allocated":118804480,"data_stored":118555300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:11.053 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:11.368 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:11.369 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:11.380 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-04-26T00:35:10.573441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20025,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516972,"kb_used_data":173900,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598548,"statfs":{"total":289910292480,"available":288356913152,"internally_reserved":0,"allocated":178073600,"data_stored":177272109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1556,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001388"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28564,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918388+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:07.918388+0000","last_peered":"2024-04-26T00:35:07.918388+0000","last_clean":"2024-04-26T00:35:07.918388+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:07.918388+0000","last_undegraded":"2024-04-26T00:35:07.918388+0000","last_fullsized":"2024-04-26T00:35:07.918388+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28312,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918650+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:07.918650+0000","last_peered":"2024-04-26T00:35:07.918650+0000","last_clean":"2024-04-26T00:35:07.918650+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:07.918650+0000","last_undegraded":"2024-04-26T00:35:07.918650+0000","last_fullsized":"2024-04-26T00:35:07.918650+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:07.918034+0000","last_peered":"2024-04-26T00:35:07.918034+0000","last_clean":"2024-04-26T00:35:07.918034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:07.918034+0000","last_undegraded":"2024-04-26T00:35:07.918034+0000","last_fullsized":"2024-04-26T00:35:07.918034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918296+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:07.918296+0000","last_peered":"2024-04-26T00:35:07.918296+0000","last_clean":"2024-04-26T00:35:07.918296+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:07.918296+0000","last_undegraded":"2024-04-26T00:35:07.918296+0000","last_fullsized":"2024-04-26T00:35:07.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918288+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:07.918288+0000","last_peered":"2024-04-26T00:35:07.918288+0000","last_clean":"2024-04-26T00:35:07.918288+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:07.918288+0000","last_undegraded":"2024-04-26T00:35:07.918288+0000","last_fullsized":"2024-04-26T00:35:07.918288+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918500+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:07.918500+0000","last_peered":"2024-04-26T00:35:07.918500+0000","last_clean":"2024-04-26T00:35:07.918500+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:07.918500+0000","last_undegraded":"2024-04-26T00:35:07.918500+0000","last_fullsized":"2024-04-26T00:35:07.918500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918614+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:07.918614+0000","last_peered":"2024-04-26T00:35:07.918614+0000","last_clean":"2024-04-26T00:35:07.918614+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:07.918614+0000","last_undegraded":"2024-04-26T00:35:07.918614+0000","last_fullsized":"2024-04-26T00:35:07.918614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918179+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:07.918179+0000","last_peered":"2024-04-26T00:35:07.918179+0000","last_clean":"2024-04-26T00:35:07.918179+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:07.918179+0000","last_undegraded":"2024-04-26T00:35:07.918179+0000","last_fullsized":"2024-04-26T00:35:07.918179+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20025,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483300,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888540,"statfs":{"total":96636764160,"available":96141864960,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280642,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084415,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:11.382 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:11.680 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:11.680 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:11.692 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-04-26T00:35:10.573441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20025,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516972,"kb_used_data":173900,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598548,"statfs":{"total":289910292480,"available":288356913152,"internally_reserved":0,"allocated":178073600,"data_stored":177272109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1556,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001388"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28564,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918388+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:07.918388+0000","last_peered":"2024-04-26T00:35:07.918388+0000","last_clean":"2024-04-26T00:35:07.918388+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:07.918388+0000","last_undegraded":"2024-04-26T00:35:07.918388+0000","last_fullsized":"2024-04-26T00:35:07.918388+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28312,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918650+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:07.918650+0000","last_peered":"2024-04-26T00:35:07.918650+0000","last_clean":"2024-04-26T00:35:07.918650+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:07.918650+0000","last_undegraded":"2024-04-26T00:35:07.918650+0000","last_fullsized":"2024-04-26T00:35:07.918650+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:07.918034+0000","last_peered":"2024-04-26T00:35:07.918034+0000","last_clean":"2024-04-26T00:35:07.918034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:07.918034+0000","last_undegraded":"2024-04-26T00:35:07.918034+0000","last_fullsized":"2024-04-26T00:35:07.918034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918296+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:07.918296+0000","last_peered":"2024-04-26T00:35:07.918296+0000","last_clean":"2024-04-26T00:35:07.918296+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:07.918296+0000","last_undegraded":"2024-04-26T00:35:07.918296+0000","last_fullsized":"2024-04-26T00:35:07.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918288+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:07.918288+0000","last_peered":"2024-04-26T00:35:07.918288+0000","last_clean":"2024-04-26T00:35:07.918288+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:07.918288+0000","last_undegraded":"2024-04-26T00:35:07.918288+0000","last_fullsized":"2024-04-26T00:35:07.918288+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918500+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:07.918500+0000","last_peered":"2024-04-26T00:35:07.918500+0000","last_clean":"2024-04-26T00:35:07.918500+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:07.918500+0000","last_undegraded":"2024-04-26T00:35:07.918500+0000","last_fullsized":"2024-04-26T00:35:07.918500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918614+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:07.918614+0000","last_peered":"2024-04-26T00:35:07.918614+0000","last_clean":"2024-04-26T00:35:07.918614+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:07.918614+0000","last_undegraded":"2024-04-26T00:35:07.918614+0000","last_fullsized":"2024-04-26T00:35:07.918614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918179+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:07.918179+0000","last_peered":"2024-04-26T00:35:07.918179+0000","last_clean":"2024-04-26T00:35:07.918179+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:07.918179+0000","last_undegraded":"2024-04-26T00:35:07.918179+0000","last_fullsized":"2024-04-26T00:35:07.918179+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20025,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483300,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888540,"statfs":{"total":96636764160,"available":96141864960,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280642,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084415,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:11.693 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-04-26T00:35:11.693 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'21260", 'reported_seq': 21913, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.081144+0000', 'last_change': '2024-04-26T00:26:01.837535+0000', 'last_active': '2024-04-26T00:34:57.081144+0000', 'last_peered': '2024-04-26T00:34:57.081144+0000', 'last_clean': '2024-04-26T00:25:59.337291+0000', 'last_became_active': '2024-04-26T00:26:01.832792+0000', 'last_became_peered': '2024-04-26T00:26:01.832792+0000', 'last_unstale': '2024-04-26T00:34:57.081144+0000', 'last_undegraded': '2024-04-26T00:26:01.450456+0000', 'last_fullsized': '2024-04-26T00:26:01.360942+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3344, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21260, 'num_write_kb': 42520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 287, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3344}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22062, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.041276+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:34:57.041276+0000', 'last_peered': '2024-04-26T00:34:57.041276+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:34:57.041276+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 21638, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.915277+0000', 'last_change': '2024-04-26T00:26:01.842290+0000', 'last_active': '2024-04-26T00:35:08.915277+0000', 'last_peered': '2024-04-26T00:35:08.915277+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:08.915277+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22199, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:56.986322+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:34:56.986322+0000', 'last_peered': '2024-04-26T00:34:56.986322+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:34:56.986322+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22034, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.009637+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:34:57.009637+0000', 'last_peered': '2024-04-26T00:34:57.009637+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:34:57.009637+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:11.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21811, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.022726+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:34:57.022726+0000', 'last_peered': '2024-04-26T00:34:57.022726+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:34:57.022726+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:11.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:11.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22142, 'reported_epoch': 44, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:34:57.069595+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:34:57.069595+0000', 'last_peered': '2024-04-26T00:34:57.069595+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:34:57.069595+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:11.695 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:12.087 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:12.087 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23186/6 objects degraded (386433.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":658,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1553379328,"bytes_avail":288356913152,"bytes_total":289910292480,"degraded_objects":23186,"degraded_total":6,"degraded_ratio":3864.3333333333339,"recovering_objects_per_sec":194,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1556,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-04-26T00:34:52.570180+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:35:12.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:12.102 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:12.415 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:12.416 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:12.429 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-04-26T00:35:10.573441+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20025,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1516972,"kb_used_data":173900,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598548,"statfs":{"total":289910292480,"available":288356913152,"internally_reserved":0,"allocated":178073600,"data_stored":177272109,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1556,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001388"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":21913,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.081144+0000","last_change":"2024-04-26T00:26:01.837535+0000","last_active":"2024-04-26T00:34:57.081144+0000","last_peered":"2024-04-26T00:34:57.081144+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:34:57.081144+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3344,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":287,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3344}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28564,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918388+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:07.918388+0000","last_peered":"2024-04-26T00:35:07.918388+0000","last_clean":"2024-04-26T00:35:07.918388+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:07.918388+0000","last_undegraded":"2024-04-26T00:35:07.918388+0000","last_fullsized":"2024-04-26T00:35:07.918388+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":21997,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058170+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:34:57.058170+0000","last_peered":"2024-04-26T00:34:57.058170+0000","last_clean":"2024-04-26T00:34:57.058170+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:34:57.058170+0000","last_undegraded":"2024-04-26T00:34:57.058170+0000","last_fullsized":"2024-04-26T00:34:57.058170+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22062,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.041276+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:34:57.041276+0000","last_peered":"2024-04-26T00:34:57.041276+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:34:57.041276+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28312,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918650+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:07.918650+0000","last_peered":"2024-04-26T00:35:07.918650+0000","last_clean":"2024-04-26T00:35:07.918650+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:07.918650+0000","last_undegraded":"2024-04-26T00:35:07.918650+0000","last_fullsized":"2024-04-26T00:35:07.918650+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21617,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:34:57.058261+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:34:57.058261+0000","last_peered":"2024-04-26T00:34:57.058261+0000","last_clean":"2024-04-26T00:34:57.058261+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:34:57.058261+0000","last_undegraded":"2024-04-26T00:34:57.058261+0000","last_fullsized":"2024-04-26T00:34:57.058261+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:07.918034+0000","last_peered":"2024-04-26T00:35:07.918034+0000","last_clean":"2024-04-26T00:35:07.918034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:07.918034+0000","last_undegraded":"2024-04-26T00:35:07.918034+0000","last_fullsized":"2024-04-26T00:35:07.918034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22199,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:56.986322+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:34:56.986322+0000","last_peered":"2024-04-26T00:34:56.986322+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:34:56.986322+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":44,"state":"active+clean","last_fresh":"2024-04-26T00:33:56.412846+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:33:56.412846+0000","last_peered":"2024-04-26T00:33:56.412846+0000","last_clean":"2024-04-26T00:33:56.412846+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:33:56.412846+0000","last_undegraded":"2024-04-26T00:33:56.412846+0000","last_fullsized":"2024-04-26T00:33:56.412846+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918296+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:07.918296+0000","last_peered":"2024-04-26T00:35:07.918296+0000","last_clean":"2024-04-26T00:35:07.918296+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:07.918296+0000","last_undegraded":"2024-04-26T00:35:07.918296+0000","last_fullsized":"2024-04-26T00:35:07.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22034,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.009637+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:34:57.009637+0000","last_peered":"2024-04-26T00:34:57.009637+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:34:57.009637+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918288+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:07.918288+0000","last_peered":"2024-04-26T00:35:07.918288+0000","last_clean":"2024-04-26T00:35:07.918288+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:07.918288+0000","last_undegraded":"2024-04-26T00:35:07.918288+0000","last_fullsized":"2024-04-26T00:35:07.918288+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21811,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.022726+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:34:57.022726+0000","last_peered":"2024-04-26T00:34:57.022726+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:34:57.022726+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22142,"reported_epoch":44,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:34:57.069595+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:34:57.069595+0000","last_peered":"2024-04-26T00:34:57.069595+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:34:57.069595+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918500+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:07.918500+0000","last_peered":"2024-04-26T00:35:07.918500+0000","last_clean":"2024-04-26T00:35:07.918500+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:07.918500+0000","last_undegraded":"2024-04-26T00:35:07.918500+0000","last_fullsized":"2024-04-26T00:35:07.918500+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918614+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:07.918614+0000","last_peered":"2024-04-26T00:35:07.918614+0000","last_clean":"2024-04-26T00:35:07.918614+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:07.918614+0000","last_undegraded":"2024-04-26T00:35:07.918614+0000","last_fullsized":"2024-04-26T00:35:07.918614+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28740,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-04-26T00:35:07.918179+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:07.918179+0000","last_peered":"2024-04-26T00:35:07.918179+0000","last_clean":"2024-04-26T00:35:07.918179+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:07.918179+0000","last_undegraded":"2024-04-26T00:35:07.918179+0000","last_fullsized":"2024-04-26T00:35:07.918179+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23186,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20025,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483300,"kb_used_data":412,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888540,"statfs":{"total":96636764160,"available":96141864960,"internally_reserved":0,"allocated":421888,"data_stored":129352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280642,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548596,"kb_used_data":57452,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823244,"statfs":{"total":96636764160,"available":96075001856,"internally_reserved":0,"allocated":58830848,"data_stored":58581668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084415,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:15.432 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:15.732 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:15.733 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:15.746 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-04-26T00:35:14.574130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20750,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525332,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590188,"statfs":{"total":289910292480,"available":288348352512,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001430"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":23368,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:12.297597+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:12.297597+0000","last_peered":"2024-04-26T00:35:12.297597+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:12.297597+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2619}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22203,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923048+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923048+0000","last_peered":"2024-04-26T00:35:08.923048+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923048+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":85,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922894+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922894+0000","last_peered":"2024-04-26T00:35:08.922894+0000","last_clean":"2024-04-26T00:35:08.922894+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922894+0000","last_undegraded":"2024-04-26T00:35:08.922894+0000","last_fullsized":"2024-04-26T00:35:08.922894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20750,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491636,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880204,"statfs":{"total":96636764160,"available":96133328896,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280643,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084415,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:15.747 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:16.093 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:16.094 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:16.106 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":370,"stamp":"2024-04-26T00:35:14.574130+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20750,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525332,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590188,"statfs":{"total":289910292480,"available":288348352512,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001430"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":23368,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:12.297597+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:12.297597+0000","last_peered":"2024-04-26T00:35:12.297597+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:12.297597+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2619}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22203,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923048+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923048+0000","last_peered":"2024-04-26T00:35:08.923048+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923048+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":85,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922894+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922894+0000","last_peered":"2024-04-26T00:35:08.922894+0000","last_clean":"2024-04-26T00:35:08.922894+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922894+0000","last_undegraded":"2024-04-26T00:35:08.922894+0000","last_fullsized":"2024-04-26T00:35:08.922894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20750,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491636,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880204,"statfs":{"total":96636764160,"available":96133328896,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280643,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084415,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:16.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-04-26T00:35:16.107 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'21260", 'reported_seq': 23368, 'reported_epoch': 46, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:12.297597+0000', 'last_change': '2024-04-26T00:35:07.689324+0000', 'last_active': '2024-04-26T00:35:12.297597+0000', 'last_peered': '2024-04-26T00:35:12.297597+0000', 'last_clean': '2024-04-26T00:25:59.337291+0000', 'last_became_active': '2024-04-26T00:26:01.832792+0000', 'last_became_peered': '2024-04-26T00:26:01.832792+0000', 'last_unstale': '2024-04-26T00:35:12.297597+0000', 'last_undegraded': '2024-04-26T00:26:01.450456+0000', 'last_fullsized': '2024-04-26T00:26:01.360942+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2619, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21260, 'num_write_kb': 42520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1012, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2619}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22066, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.922985+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:08.922985+0000', 'last_peered': '2024-04-26T00:35:08.922985+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:08.922985+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 21638, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.915277+0000', 'last_change': '2024-04-26T00:26:01.842290+0000', 'last_active': '2024-04-26T00:35:08.915277+0000', 'last_peered': '2024-04-26T00:35:08.915277+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:08.915277+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22203, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.923048+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:35:08.923048+0000', 'last_peered': '2024-04-26T00:35:08.923048+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:08.923048+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22038, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.923122+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:08.923122+0000', 'last_peered': '2024-04-26T00:35:08.923122+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:08.923122+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:16.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:16.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21815, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.924750+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:08.924750+0000', 'last_peered': '2024-04-26T00:35:08.924750+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:08.924750+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:16.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:16.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22146, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.924625+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:08.924625+0000', 'last_peered': '2024-04-26T00:35:08.924625+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:08.924625+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:16.109 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:16.484 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:16.484 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22461/6 objects degraded (374350.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":662,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1561939968,"bytes_avail":288348352512,"bytes_total":289910292480,"degraded_objects":22461,"degraded_total":6,"degraded_ratio":3743.5,"recovering_objects_per_sec":162,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":12,"modified":"2024-04-26T00:34:52.570180+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T00:35:16.497 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:16.497 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:16.798 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:16.798 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:16.810 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-04-26T00:35:16.574475+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20750,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525332,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590188,"statfs":{"total":289910292480,"available":288348352512,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.665875"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":23368,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:12.297597+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:12.297597+0000","last_peered":"2024-04-26T00:35:12.297597+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:12.297597+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2619}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22203,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923048+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923048+0000","last_peered":"2024-04-26T00:35:08.923048+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923048+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":85,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922894+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922894+0000","last_peered":"2024-04-26T00:35:08.922894+0000","last_clean":"2024-04-26T00:35:08.922894+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922894+0000","last_undegraded":"2024-04-26T00:35:08.922894+0000","last_fullsized":"2024-04-26T00:35:08.922894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20750,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491636,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880204,"statfs":{"total":96636764160,"available":96133328896,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280643,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:19.812 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:20.113 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:20.113 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:20.125 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-04-26T00:35:18.574804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21369,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512852,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281602668,"statfs":{"total":289910292480,"available":288361132032,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1370999916},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1919,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001691"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":24606,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:17.295185+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:17.295185+0000","last_peered":"2024-04-26T00:35:17.295185+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:17.295185+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22203,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923048+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923048+0000","last_peered":"2024-04-26T00:35:08.923048+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923048+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":85,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922894+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922894+0000","last_peered":"2024-04-26T00:35:08.922894+0000","last_clean":"2024-04-26T00:35:08.922894+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922894+0000","last_undegraded":"2024-04-26T00:35:08.922894+0000","last_fullsized":"2024-04-26T00:35:08.922894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21369,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479156,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892684,"statfs":{"total":96636764160,"available":96146108416,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":490204879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280644,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:20.126 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:20.422 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:20.423 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:20.435 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-04-26T00:35:18.574804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21369,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512852,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281602668,"statfs":{"total":289910292480,"available":288361132032,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1370999916},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1919,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001691"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":24606,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:17.295185+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:17.295185+0000","last_peered":"2024-04-26T00:35:17.295185+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:17.295185+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22203,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923048+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923048+0000","last_peered":"2024-04-26T00:35:08.923048+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923048+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":85,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922894+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922894+0000","last_peered":"2024-04-26T00:35:08.922894+0000","last_clean":"2024-04-26T00:35:08.922894+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922894+0000","last_undegraded":"2024-04-26T00:35:08.922894+0000","last_fullsized":"2024-04-26T00:35:08.922894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21369,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479156,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892684,"statfs":{"total":96636764160,"available":96146108416,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":490204879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280644,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084416,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:20.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-04-26T00:35:20.437 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'21260", 'reported_seq': 24606, 'reported_epoch': 46, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:17.295185+0000', 'last_change': '2024-04-26T00:35:07.689324+0000', 'last_active': '2024-04-26T00:35:17.295185+0000', 'last_peered': '2024-04-26T00:35:17.295185+0000', 'last_clean': '2024-04-26T00:25:59.337291+0000', 'last_became_active': '2024-04-26T00:26:01.832792+0000', 'last_became_peered': '2024-04-26T00:26:01.832792+0000', 'last_unstale': '2024-04-26T00:35:17.295185+0000', 'last_undegraded': '2024-04-26T00:26:01.450456+0000', 'last_fullsized': '2024-04-26T00:26:01.360942+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2000, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21260, 'num_write_kb': 42520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1631, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2000}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:20.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:20.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22066, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.922985+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:08.922985+0000', 'last_peered': '2024-04-26T00:35:08.922985+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:08.922985+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:20.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:20.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 21638, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.915277+0000', 'last_change': '2024-04-26T00:26:01.842290+0000', 'last_active': '2024-04-26T00:35:08.915277+0000', 'last_peered': '2024-04-26T00:35:08.915277+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:08.915277+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:20.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:20.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22203, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.923048+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:35:08.923048+0000', 'last_peered': '2024-04-26T00:35:08.923048+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:08.923048+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:20.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:20.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22038, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.923122+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:08.923122+0000', 'last_peered': '2024-04-26T00:35:08.923122+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:08.923122+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:20.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:20.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21815, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.924750+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:08.924750+0000', 'last_peered': '2024-04-26T00:35:08.924750+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:08.924750+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:20.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:20.440 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22146, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.924625+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:08.924625+0000', 'last_peered': '2024-04-26T00:35:08.924625+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:08.924625+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:20.440 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:20.849 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:20.849 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21842/6 objects degraded (364033.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":666,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1549160448,"bytes_avail":288361132032,"bytes_total":289910292480,"degraded_objects":21842,"degraded_total":6,"degraded_ratio":3640.3333333333339,"recovering_objects_per_sec":191,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1919,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:20.861 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:20.862 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:21.163 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:21.163 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:21.175 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":373,"stamp":"2024-04-26T00:35:20.575207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21369,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512852,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281602668,"statfs":{"total":289910292480,"available":288361132032,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1370999916},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1919,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1919,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.662049"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":24606,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:17.295185+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:17.295185+0000","last_peered":"2024-04-26T00:35:17.295185+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:17.295185+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2000,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2000}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22203,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923048+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923048+0000","last_peered":"2024-04-26T00:35:08.923048+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923048+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":85,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922894+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922894+0000","last_peered":"2024-04-26T00:35:08.922894+0000","last_clean":"2024-04-26T00:35:08.922894+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922894+0000","last_undegraded":"2024-04-26T00:35:08.922894+0000","last_fullsized":"2024-04-26T00:35:08.922894+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21842,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21369,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479156,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892684,"statfs":{"total":96636764160,"available":96146108416,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":490204879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280644,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084417,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:24.177 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:24.479 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:24.479 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:24.494 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-04-26T00:35:22.575517+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22220,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512852,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1338867,"kb_avail":281602668,"statfs":{"total":289910292480,"available":288361132032,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1370999916},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2195,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2195,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002075"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":26308,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:22.297859+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:22.297859+0000","last_peered":"2024-04-26T00:35:22.297859+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:22.297859+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1149}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923924+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923924+0000","last_peered":"2024-04-26T00:35:08.923924+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923924+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":86,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922931+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922931+0000","last_peered":"2024-04-26T00:35:08.922931+0000","last_clean":"2024-04-26T00:35:08.922931+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922931+0000","last_undegraded":"2024-04-26T00:35:08.922931+0000","last_fullsized":"2024-04-26T00:35:08.922931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22220,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479156,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":478715,"kb_avail":93892684,"statfs":{"total":96636764160,"available":96146108416,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":490204879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280645,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084417,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:24.495 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:24.806 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:24.807 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:24.819 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-04-26T00:35:24.575853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22220,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517012,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598508,"statfs":{"total":289910292480,"available":288356872192,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002050"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":26308,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:22.297859+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:22.297859+0000","last_peered":"2024-04-26T00:35:22.297859+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:22.297859+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1149}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923924+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923924+0000","last_peered":"2024-04-26T00:35:08.923924+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923924+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":86,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922931+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922931+0000","last_peered":"2024-04-26T00:35:08.922931+0000","last_clean":"2024-04-26T00:35:08.922931+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922931+0000","last_undegraded":"2024-04-26T00:35:08.922931+0000","last_fullsized":"2024-04-26T00:35:08.922931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22220,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483316,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888524,"statfs":{"total":96636764160,"available":96141848576,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280645,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084417,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:24.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-04-26T00:35:24.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'21260", 'reported_seq': 26308, 'reported_epoch': 46, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:22.297859+0000', 'last_change': '2024-04-26T00:35:07.689324+0000', 'last_active': '2024-04-26T00:35:22.297859+0000', 'last_peered': '2024-04-26T00:35:22.297859+0000', 'last_clean': '2024-04-26T00:25:59.337291+0000', 'last_became_active': '2024-04-26T00:26:01.832792+0000', 'last_became_peered': '2024-04-26T00:26:01.832792+0000', 'last_unstale': '2024-04-26T00:35:22.297859+0000', 'last_undegraded': '2024-04-26T00:26:01.450456+0000', 'last_fullsized': '2024-04-26T00:26:01.360942+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1149, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21260, 'num_write_kb': 42520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2482, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1149}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:24.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:24.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22066, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.922985+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:08.922985+0000', 'last_peered': '2024-04-26T00:35:08.922985+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:08.922985+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:24.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:24.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 21638, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.915277+0000', 'last_change': '2024-04-26T00:26:01.842290+0000', 'last_active': '2024-04-26T00:35:08.915277+0000', 'last_peered': '2024-04-26T00:35:08.915277+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:08.915277+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:24.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:24.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22204, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.923924+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:35:08.923924+0000', 'last_peered': '2024-04-26T00:35:08.923924+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:08.923924+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:24.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:24.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22038, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.923122+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:08.923122+0000', 'last_peered': '2024-04-26T00:35:08.923122+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:08.923122+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:24.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:24.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21815, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.924750+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:08.924750+0000', 'last_peered': '2024-04-26T00:35:08.924750+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:08.924750+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:24.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:24.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22146, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.924625+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:08.924625+0000', 'last_peered': '2024-04-26T00:35:08.924625+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:08.924625+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:24.821 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:25.191 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:25.191 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20991/6 objects degraded (349850.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":671,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":46,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1553420288,"bytes_avail":288356872192,"bytes_total":289910292480,"degraded_objects":20991,"degraded_total":6,"degraded_ratio":3498.5,"recovering_objects_per_sec":122,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:25.204 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:25.205 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:26.315 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:26.316 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:26.328 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-04-26T00:35:24.575853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22220,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517012,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598508,"statfs":{"total":289910292480,"available":288356872192,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1470,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1470,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002050"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":26308,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:22.297859+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:22.297859+0000","last_peered":"2024-04-26T00:35:22.297859+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:22.297859+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1149,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2482,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1149}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923924+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923924+0000","last_peered":"2024-04-26T00:35:08.923924+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923924+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":86,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922931+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922931+0000","last_peered":"2024-04-26T00:35:08.922931+0000","last_clean":"2024-04-26T00:35:08.922931+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922931+0000","last_undegraded":"2024-04-26T00:35:08.922931+0000","last_fullsized":"2024-04-26T00:35:08.922931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20991,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22220,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176951296,"data_stored":176897101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483316,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888524,"statfs":{"total":96636764160,"available":96141848576,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280645,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084417,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":6279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:29.331 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:29.628 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:29.628 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:29.640 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-04-26T00:35:28.947334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23274,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525332,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590188,"statfs":{"total":289910292480,"available":288348352512,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1905,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1905,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.372127"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28416,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:27.300758+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:27.300758+0000","last_peered":"2024-04-26T00:35:27.300758+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:27.300758+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":95,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-95}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923924+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923924+0000","last_peered":"2024-04-26T00:35:08.923924+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923924+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":86,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922931+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922931+0000","last_peered":"2024-04-26T00:35:08.922931+0000","last_clean":"2024-04-26T00:35:08.922931+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922931+0000","last_undegraded":"2024-04-26T00:35:08.922931+0000","last_fullsized":"2024-04-26T00:35:08.922931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23274,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491636,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880204,"statfs":{"total":96636764160,"available":96133328896,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280646,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084418,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:29.642 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:29.935 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:29.936 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:29.948 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-04-26T00:35:28.947334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23274,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525332,"kb_used_data":173932,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590188,"statfs":{"total":289910292480,"available":288348352512,"internally_reserved":0,"allocated":178106368,"data_stored":177283687,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1905,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1905,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.372127"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28416,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:27.300758+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:27.300758+0000","last_peered":"2024-04-26T00:35:27.300758+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:27.300758+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":95,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-95}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21783,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915526+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:08.915526+0000","last_peered":"2024-04-26T00:35:08.915526+0000","last_clean":"2024-04-26T00:35:08.915526+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:08.915526+0000","last_undegraded":"2024-04-26T00:35:08.915526+0000","last_fullsized":"2024-04-26T00:35:08.915526+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21155,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915809+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:08.915809+0000","last_peered":"2024-04-26T00:35:08.915809+0000","last_clean":"2024-04-26T00:35:08.915809+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:08.915809+0000","last_undegraded":"2024-04-26T00:35:08.915809+0000","last_fullsized":"2024-04-26T00:35:08.915809+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21638,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.915277+0000","last_change":"2024-04-26T00:26:01.842290+0000","last_active":"2024-04-26T00:35:08.915277+0000","last_peered":"2024-04-26T00:35:08.915277+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:08.915277+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3285,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":182,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3281}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914919+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:08.914919+0000","last_peered":"2024-04-26T00:35:08.914919+0000","last_clean":"2024-04-26T00:35:08.914919+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:08.914919+0000","last_undegraded":"2024-04-26T00:35:08.914919+0000","last_fullsized":"2024-04-26T00:35:08.914919+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923924+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923924+0000","last_peered":"2024-04-26T00:35:08.923924+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923924+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":86,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922931+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922931+0000","last_peered":"2024-04-26T00:35:08.922931+0000","last_clean":"2024-04-26T00:35:08.922931+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922931+0000","last_undegraded":"2024-04-26T00:35:08.922931+0000","last_fullsized":"2024-04-26T00:35:08.922931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28893,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:09.008840+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:09.008840+0000","last_peered":"2024-04-26T00:35:09.008840+0000","last_clean":"2024-04-26T00:35:09.008840+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:09.008840+0000","last_undegraded":"2024-04-26T00:35:09.008840+0000","last_fullsized":"2024-04-26T00:35:09.008840+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28328,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.914904+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:08.914904+0000","last_peered":"2024-04-26T00:35:08.914904+0000","last_clean":"2024-04-26T00:35:08.914904+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:08.914904+0000","last_undegraded":"2024-04-26T00:35:08.914904+0000","last_fullsized":"2024-04-26T00:35:08.914904+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.915837+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:08.915837+0000","last_peered":"2024-04-26T00:35:08.915837+0000","last_clean":"2024-04-26T00:35:08.915837+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:08.915837+0000","last_undegraded":"2024-04-26T00:35:08.915837+0000","last_fullsized":"2024-04-26T00:35:08.915837+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19937,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23274,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491636,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880204,"statfs":{"total":96636764160,"available":96133328896,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280646,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084418,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485076,"kb_used_data":116036,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886764,"statfs":{"total":96636764160,"available":96140046336,"internally_reserved":0,"allocated":118820864,"data_stored":118561089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:29.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-04-26T00:35:29.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "44'21260", 'reported_seq': 28416, 'reported_epoch': 46, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:27.300758+0000', 'last_change': '2024-04-26T00:35:07.689324+0000', 'last_active': '2024-04-26T00:35:27.300758+0000', 'last_peered': '2024-04-26T00:35:27.300758+0000', 'last_clean': '2024-04-26T00:25:59.337291+0000', 'last_became_active': '2024-04-26T00:26:01.832792+0000', 'last_became_peered': '2024-04-26T00:26:01.832792+0000', 'last_unstale': '2024-04-26T00:35:27.300758+0000', 'last_undegraded': '2024-04-26T00:26:01.450456+0000', 'last_fullsized': '2024-04-26T00:26:01.360942+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 0, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 95, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 21260, 'num_write_kb': 42520, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3536, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -95}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:29.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:29.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22066, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.922985+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:08.922985+0000', 'last_peered': '2024-04-26T00:35:08.922985+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:08.922985+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 21638, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.915277+0000', 'last_change': '2024-04-26T00:26:01.842290+0000', 'last_active': '2024-04-26T00:35:08.915277+0000', 'last_peered': '2024-04-26T00:35:08.915277+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:08.915277+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3285, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 182, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22204, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.923924+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:35:08.923924+0000', 'last_peered': '2024-04-26T00:35:08.923924+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:08.923924+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22038, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.923122+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:08.923122+0000', 'last_peered': '2024-04-26T00:35:08.923122+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:08.923122+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21815, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.924750+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:08.924750+0000', 'last_peered': '2024-04-26T00:35:08.924750+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:08.924750+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:29.950 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22146, 'reported_epoch': 46, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:08.924625+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:08.924625+0000', 'last_peered': '2024-04-26T00:35:08.924625+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:08.924625+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:29.951 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:30.379 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:30.379 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19937/6 objects degraded (332283.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":676,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1561939968,"bytes_avail":288348352512,"bytes_total":289910292480,"degraded_objects":19937,"degraded_total":6,"degraded_ratio":3322.8333333333339,"recovering_objects_per_sec":190,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1905,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:30.392 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:30.392 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:30.699 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:30.699 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:30.711 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-04-26T00:35:30.576798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23552,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525348,"kb_used_data":173948,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590172,"statfs":{"total":289910292480,"available":288348336128,"internally_reserved":0,"allocated":178122752,"data_stored":177289426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1332,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1332,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001280"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28416,"reported_epoch":46,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:27.300758+0000","last_change":"2024-04-26T00:35:07.689324+0000","last_active":"2024-04-26T00:35:27.300758+0000","last_peered":"2024-04-26T00:35:27.300758+0000","last_clean":"2024-04-26T00:25:59.337291+0000","last_became_active":"2024-04-26T00:26:01.832792+0000","last_became_peered":"2024-04-26T00:26:01.832792+0000","last_unstale":"2024-04-26T00:35:27.300758+0000","last_undegraded":"2024-04-26T00:26:01.450456+0000","last_fullsized":"2024-04-26T00:26:01.360942+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":95,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3536,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-95}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22001,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.923715+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:08.923715+0000","last_peered":"2024-04-26T00:35:08.923715+0000","last_clean":"2024-04-26T00:35:08.923715+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:08.923715+0000","last_undegraded":"2024-04-26T00:35:08.923715+0000","last_fullsized":"2024-04-26T00:35:08.923715+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22066,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.922985+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:08.922985+0000","last_peered":"2024-04-26T00:35:08.922985+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:08.922985+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21621,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922802+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:08.922802+0000","last_peered":"2024-04-26T00:35:08.922802+0000","last_clean":"2024-04-26T00:35:08.922802+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:08.922802+0000","last_undegraded":"2024-04-26T00:35:08.922802+0000","last_fullsized":"2024-04-26T00:35:08.922802+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":22199,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:30.005235+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:30.005235+0000","last_peered":"2024-04-26T00:35:30.005235+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:30.005235+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22204,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923924+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:08.923924+0000","last_peered":"2024-04-26T00:35:08.923924+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:08.923924+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":86,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.922931+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:08.922931+0000","last_peered":"2024-04-26T00:35:08.922931+0000","last_clean":"2024-04-26T00:35:08.922931+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:08.922931+0000","last_undegraded":"2024-04-26T00:35:08.922931+0000","last_fullsized":"2024-04-26T00:35:08.922931+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22038,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.923122+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:08.923122+0000","last_peered":"2024-04-26T00:35:08.923122+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:08.923122+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21815,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924750+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:08.924750+0000","last_peered":"2024-04-26T00:35:08.924750+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:08.924750+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948322+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948322+0000","last_peered":"2024-04-26T00:35:28.948322+0000","last_clean":"2024-04-26T00:35:28.948322+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948322+0000","last_undegraded":"2024-04-26T00:35:28.948322+0000","last_fullsized":"2024-04-26T00:35:28.948322+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22146,"reported_epoch":46,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:08.924625+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:08.924625+0000","last_peered":"2024-04-26T00:35:08.924625+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:08.924625+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23552,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491636,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880204,"statfs":{"total":96636764160,"available":96133328896,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280646,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548620,"kb_used_data":57468,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823220,"statfs":{"total":96636764160,"available":96074977280,"internally_reserved":0,"allocated":58847232,"data_stored":58587457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084419,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:33.713 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:34.020 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:34.020 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:34.033 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-04-26T00:35:32.577119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23647,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525364,"kb_used_data":173964,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590156,"statfs":{"total":289910292480,"available":288348319744,"internally_reserved":0,"allocated":178139136,"data_stored":177295165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001265"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":22199,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:30.005235+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:30.005235+0000","last_peered":"2024-04-26T00:35:30.005235+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:30.005235+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948322+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948322+0000","last_peered":"2024-04-26T00:35:28.948322+0000","last_clean":"2024-04-26T00:35:28.948322+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948322+0000","last_undegraded":"2024-04-26T00:35:28.948322+0000","last_fullsized":"2024-04-26T00:35:28.948322+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23647,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491636,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880204,"statfs":{"total":96636764160,"available":96133328896,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280647,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084419,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:34.035 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:34.358 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:34.359 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:34.371 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-04-26T00:35:32.577119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23647,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1525364,"kb_used_data":173964,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281590156,"statfs":{"total":289910292480,"available":288348319744,"internally_reserved":0,"allocated":178139136,"data_stored":177295165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001265"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28566,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918142+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:08.918142+0000","last_peered":"2024-04-26T00:35:08.918142+0000","last_clean":"2024-04-26T00:35:08.918142+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:08.918142+0000","last_undegraded":"2024-04-26T00:35:08.918142+0000","last_fullsized":"2024-04-26T00:35:08.918142+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28314,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.919184+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:08.919184+0000","last_peered":"2024-04-26T00:35:08.919184+0000","last_clean":"2024-04-26T00:35:08.919184+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:08.919184+0000","last_undegraded":"2024-04-26T00:35:08.919184+0000","last_fullsized":"2024-04-26T00:35:08.919184+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.917822+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:08.917822+0000","last_peered":"2024-04-26T00:35:08.917822+0000","last_clean":"2024-04-26T00:35:08.917822+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:08.917822+0000","last_undegraded":"2024-04-26T00:35:08.917822+0000","last_fullsized":"2024-04-26T00:35:08.917822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":22199,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:30.005235+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:30.005235+0000","last_peered":"2024-04-26T00:35:30.005235+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:30.005235+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918045+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:08.918045+0000","last_peered":"2024-04-26T00:35:08.918045+0000","last_clean":"2024-04-26T00:35:08.918045+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:08.918045+0000","last_undegraded":"2024-04-26T00:35:08.918045+0000","last_fullsized":"2024-04-26T00:35:08.918045+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918220+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:08.918220+0000","last_peered":"2024-04-26T00:35:08.918220+0000","last_clean":"2024-04-26T00:35:08.918220+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:08.918220+0000","last_undegraded":"2024-04-26T00:35:08.918220+0000","last_fullsized":"2024-04-26T00:35:08.918220+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948322+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948322+0000","last_peered":"2024-04-26T00:35:28.948322+0000","last_clean":"2024-04-26T00:35:28.948322+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948322+0000","last_undegraded":"2024-04-26T00:35:28.948322+0000","last_fullsized":"2024-04-26T00:35:28.948322+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918296+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:08.918296+0000","last_peered":"2024-04-26T00:35:08.918296+0000","last_clean":"2024-04-26T00:35:08.918296+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:08.918296+0000","last_undegraded":"2024-04-26T00:35:08.918296+0000","last_fullsized":"2024-04-26T00:35:08.918296+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918321+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:08.918321+0000","last_peered":"2024-04-26T00:35:08.918321+0000","last_clean":"2024-04-26T00:35:08.918321+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:08.918321+0000","last_undegraded":"2024-04-26T00:35:08.918321+0000","last_fullsized":"2024-04-26T00:35:08.918321+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28742,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-04-26T00:35:08.918135+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:08.918135+0000","last_peered":"2024-04-26T00:35:08.918135+0000","last_clean":"2024-04-26T00:35:08.918135+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:08.918135+0000","last_undegraded":"2024-04-26T00:35:08.918135+0000","last_fullsized":"2024-04-26T00:35:08.918135+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23647,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491636,"kb_used_data":428,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880204,"statfs":{"total":96636764160,"available":96133328896,"internally_reserved":0,"allocated":438272,"data_stored":135141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280647,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084419,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:34.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:34.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22070, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949569+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:28.949569+0000', 'last_peered': '2024-04-26T00:35:28.949569+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:28.949569+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:34.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:34.372 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 22199, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:30.005235+0000', 'last_change': '2024-04-26T00:35:27.654706+0000', 'last_active': '2024-04-26T00:35:30.005235+0000', 'last_peered': '2024-04-26T00:35:30.005235+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:30.005235+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3007, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 460, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3005}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:34.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:34.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22208, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949591+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:35:28.949591+0000', 'last_peered': '2024-04-26T00:35:28.949591+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:28.949591+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:34.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:34.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22042, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949668+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:28.949668+0000', 'last_peered': '2024-04-26T00:35:28.949668+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:28.949668+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:34.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:34.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21819, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949723+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:28.949723+0000', 'last_peered': '2024-04-26T00:35:28.949723+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:28.949723+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:34.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:34.373 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22150, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.950027+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:28.950027+0000', 'last_peered': '2024-04-26T00:35:28.950027+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:28.950027+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:34.373 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:34.736 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:34.736 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19564/6 objects degraded (326066.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":680,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1561972736,"bytes_avail":288348319744,"bytes_total":289910292480,"degraded_objects":19564,"degraded_total":6,"degraded_ratio":3260.6666666666661,"recovering_objects_per_sec":178,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:34.752 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:34.752 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:35.074 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:35.075 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:35.087 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-04-26T00:35:34.577482+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23647,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517052,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598468,"statfs":{"total":289910292480,"available":288356831232,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1427,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1427,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001317"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":22199,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:30.005235+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:30.005235+0000","last_peered":"2024-04-26T00:35:30.005235+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:30.005235+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3007,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":460,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3005}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":62,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948322+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948322+0000","last_peered":"2024-04-26T00:35:28.948322+0000","last_clean":"2024-04-26T00:35:28.948322+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948322+0000","last_undegraded":"2024-04-26T00:35:28.948322+0000","last_fullsized":"2024-04-26T00:35:28.948322+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19564,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23647,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483324,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888516,"statfs":{"total":96636764160,"available":96141840384,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280647,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084419,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:38.089 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:38.388 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:38.389 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:38.400 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-04-26T00:35:36.577806+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24284,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1517052,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281598468,"statfs":{"total":289910292480,"available":288356831232,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1010,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.629461"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":23473,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:35.011213+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:35.011213+0000","last_peered":"2024-04-26T00:35:35.011213+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:35.011213+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1097,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2368}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24284,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483324,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888516,"statfs":{"total":96636764160,"available":96141840384,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280647,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084420,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:38.402 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:38.702 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:38.702 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:38.716 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-26T00:35:38.578136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24284,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521212,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281594308,"statfs":{"total":289910292480,"available":288352571392,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1010,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001659"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":23473,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:35.011213+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:35.011213+0000","last_peered":"2024-04-26T00:35:35.011213+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:35.011213+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1097,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2368}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24284,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487484,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884356,"statfs":{"total":96636764160,"available":96137580544,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":498724559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280648,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084420,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:38.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:38.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22070, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949569+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:28.949569+0000', 'last_peered': '2024-04-26T00:35:28.949569+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:28.949569+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:38.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:38.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 23473, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:35.011213+0000', 'last_change': '2024-04-26T00:35:27.654706+0000', 'last_active': '2024-04-26T00:35:35.011213+0000', 'last_peered': '2024-04-26T00:35:35.011213+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:35.011213+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2370, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1097, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2368}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:38.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:38.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22208, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949591+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:35:28.949591+0000', 'last_peered': '2024-04-26T00:35:28.949591+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:28.949591+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:38.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:38.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22042, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949668+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:28.949668+0000', 'last_peered': '2024-04-26T00:35:28.949668+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:28.949668+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:38.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:38.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21819, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949723+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:28.949723+0000', 'last_peered': '2024-04-26T00:35:28.949723+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:28.949723+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:38.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:38.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22150, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.950027+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:28.950027+0000', 'last_peered': '2024-04-26T00:35:28.950027+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:28.950027+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:38.718 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:39.121 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:39.121 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18927/6 objects degraded (315450.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":685,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1557721088,"bytes_avail":288352571392,"bytes_total":289910292480,"degraded_objects":18927,"degraded_total":6,"degraded_ratio":3154.5,"recovering_objects_per_sec":100,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1010,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:39.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:39.135 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:40.245 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:40.246 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:40.261 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":384,"stamp":"2024-04-26T00:35:38.578136+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24284,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521212,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281594308,"statfs":{"total":289910292480,"available":288352571392,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1010,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1010,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001659"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":23473,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:35.011213+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:35.011213+0000","last_peered":"2024-04-26T00:35:35.011213+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:35.011213+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2370,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1097,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2368}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18927,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24284,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487484,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884356,"statfs":{"total":96636764160,"available":96137580544,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":498724559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280648,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084420,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:43.263 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:43.575 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:43.576 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:43.588 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-26T00:35:42.578780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24963,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521212,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281594308,"statfs":{"total":289910292480,"available":288352571392,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1411,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001982"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":24831,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:40.010233+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:40.010233+0000","last_peered":"2024-04-26T00:35:40.010233+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:40.010233+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1691,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1776,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24963,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487484,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884356,"statfs":{"total":96636764160,"available":96137580544,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":498724559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280649,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084421,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:43.590 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:43.890 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:43.890 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:43.903 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":386,"stamp":"2024-04-26T00:35:42.578780+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24963,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521212,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281594308,"statfs":{"total":289910292480,"available":288352571392,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1411,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1411,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001982"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":24831,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:40.010233+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:40.010233+0000","last_peered":"2024-04-26T00:35:40.010233+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:40.010233+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1691,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1776,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24963,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487484,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884356,"statfs":{"total":96636764160,"available":96137580544,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":498724559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280649,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084421,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22070, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949569+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:28.949569+0000', 'last_peered': '2024-04-26T00:35:28.949569+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:28.949569+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 24831, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:40.010233+0000', 'last_change': '2024-04-26T00:35:27.654706+0000', 'last_active': '2024-04-26T00:35:40.010233+0000', 'last_peered': '2024-04-26T00:35:40.010233+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:40.010233+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1691, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1776, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1689}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22208, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949591+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:35:28.949591+0000', 'last_peered': '2024-04-26T00:35:28.949591+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:28.949591+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22042, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949668+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:28.949668+0000', 'last_peered': '2024-04-26T00:35:28.949668+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:28.949668+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:43.905 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21819, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949723+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:28.949723+0000', 'last_peered': '2024-04-26T00:35:28.949723+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:28.949723+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:43.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:43.906 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22150, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.950027+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:28.950027+0000', 'last_peered': '2024-04-26T00:35:28.950027+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:28.950027+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:43.906 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:44.275 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:44.275 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18248/6 objects degraded (304133.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":690,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1557721088,"bytes_avail":288352571392,"bytes_total":289910292480,"degraded_objects":18248,"degraded_total":6,"degraded_ratio":3041.3333333333339,"recovering_objects_per_sec":117,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1411,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:44.289 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:44.289 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:44.612 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:44.612 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:44.625 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-04-26T00:35:44.579093+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24963,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529532,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1355507,"kb_avail":281585988,"statfs":{"total":289910292480,"available":288344051712,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1388039276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1316,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1316,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001974"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":24831,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:40.010233+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:40.010233+0000","last_peered":"2024-04-26T00:35:40.010233+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:40.010233+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1691,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1776,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1689}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24963,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495804,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93876036,"statfs":{"total":96636764160,"available":96129060864,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":507244239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280649,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084421,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:47.627 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:47.974 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:47.985 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-04-26T00:35:46.579402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25886,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529532,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1355507,"kb_avail":281585988,"statfs":{"total":289910292480,"available":288344051712,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1388039276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001920"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":26677,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:45.013605+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:45.013605+0000","last_peered":"2024-04-26T00:35:45.013605+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:45.013605+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25886,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495804,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93876036,"statfs":{"total":96636764160,"available":96129060864,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":507244239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280649,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084422,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:47.987 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:48.282 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:48.282 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:48.293 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":388,"stamp":"2024-04-26T00:35:46.579402+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25886,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529532,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1355507,"kb_avail":281585988,"statfs":{"total":289910292480,"available":288344051712,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1388039276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001920"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":26677,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:45.013605+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:45.013605+0000","last_peered":"2024-04-26T00:35:45.013605+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:45.013605+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21819,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949723+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.949723+0000","last_peered":"2024-04-26T00:35:28.949723+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.949723+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25886,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495804,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":495355,"kb_avail":93876036,"statfs":{"total":96636764160,"available":96129060864,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":507244239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280649,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084422,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22070, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949569+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:28.949569+0000', 'last_peered': '2024-04-26T00:35:28.949569+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:28.949569+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 26677, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:45.013605+0000', 'last_change': '2024-04-26T00:35:27.654706+0000', 'last_active': '2024-04-26T00:35:45.013605+0000', 'last_peered': '2024-04-26T00:35:45.013605+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:45.013605+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 768, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2699, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -766}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 22208, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949591+0000', 'last_change': '2024-04-26T00:26:01.839861+0000', 'last_active': '2024-04-26T00:35:28.949591+0000', 'last_peered': '2024-04-26T00:35:28.949591+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:28.949591+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3302, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 322, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3300}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22042, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949668+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:28.949668+0000', 'last_peered': '2024-04-26T00:35:28.949668+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:28.949668+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:48.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21819, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.949723+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:28.949723+0000', 'last_peered': '2024-04-26T00:35:28.949723+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:28.949723+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:48.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:48.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22150, 'reported_epoch': 48, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:28.950027+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:28.950027+0000', 'last_peered': '2024-04-26T00:35:28.950027+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:28.950027+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:48.296 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:48.680 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:48.680 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17325/6 objects degraded (288750.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":694,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":48,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1566240768,"bytes_avail":288344051712,"bytes_total":289910292480,"degraded_objects":17325,"degraded_total":6,"degraded_ratio":2887.5,"recovering_objects_per_sec":186,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2239,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:48.693 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:48.693 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:49.047 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:49.048 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:49.060 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":390,"stamp":"2024-04-26T00:35:49.005710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25886,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1521212,"kb_used_data":173972,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281594308,"statfs":{"total":289910292480,"available":288352571392,"internally_reserved":0,"allocated":178147328,"data_stored":177299938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1602,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.427574"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21787,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948409+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:28.948409+0000","last_peered":"2024-04-26T00:35:28.948409+0000","last_clean":"2024-04-26T00:35:28.948409+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:28.948409+0000","last_undegraded":"2024-04-26T00:35:28.948409+0000","last_fullsized":"2024-04-26T00:35:28.948409+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21159,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948249+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:28.948249+0000","last_peered":"2024-04-26T00:35:28.948249+0000","last_clean":"2024-04-26T00:35:28.948249+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:28.948249+0000","last_undegraded":"2024-04-26T00:35:28.948249+0000","last_fullsized":"2024-04-26T00:35:28.948249+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":26677,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:45.013605+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:45.013605+0000","last_peered":"2024-04-26T00:35:45.013605+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:45.013605+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947889+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:28.947889+0000","last_peered":"2024-04-26T00:35:28.947889+0000","last_clean":"2024-04-26T00:35:28.947889+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:28.947889+0000","last_undegraded":"2024-04-26T00:35:28.947889+0000","last_fullsized":"2024-04-26T00:35:28.947889+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28898,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.947896+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:28.947896+0000","last_peered":"2024-04-26T00:35:28.947896+0000","last_clean":"2024-04-26T00:35:28.947896+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:28.947896+0000","last_undegraded":"2024-04-26T00:35:28.947896+0000","last_fullsized":"2024-04-26T00:35:28.947896+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28332,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948011+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:28.948011+0000","last_peered":"2024-04-26T00:35:28.948011+0000","last_clean":"2024-04-26T00:35:28.948011+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:28.948011+0000","last_undegraded":"2024-04-26T00:35:28.948011+0000","last_fullsized":"2024-04-26T00:35:28.948011+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21820,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950922+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.950922+0000","last_peered":"2024-04-26T00:35:28.950922+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.950922+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":63,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.948357+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:28.948357+0000","last_peered":"2024-04-26T00:35:28.948357+0000","last_clean":"2024-04-26T00:35:28.948357+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:28.948357+0000","last_undegraded":"2024-04-26T00:35:28.948357+0000","last_fullsized":"2024-04-26T00:35:28.948357+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25886,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176943104,"data_stored":176896135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487484,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884356,"statfs":{"total":96636764160,"available":96137580544,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":498724559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280650,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084422,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485092,"kb_used_data":116052,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93886748,"statfs":{"total":96636764160,"available":96140029952,"internally_reserved":0,"allocated":118837248,"data_stored":118566828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118435840,"data_stored":118432227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:52.063 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:52.394 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:52.394 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:52.407 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-04-26T00:35:50.580175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25886,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520868,"kb_used_data":173632,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281594652,"statfs":{"total":289910292480,"available":288352923648,"internally_reserved":0,"allocated":177799168,"data_stored":176946441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001395"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28616,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:29.015703+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:29.015703+0000","last_peered":"2024-04-26T00:35:29.015703+0000","last_clean":"2024-04-26T00:35:29.015703+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:29.015703+0000","last_undegraded":"2024-04-26T00:35:29.015703+0000","last_fullsized":"2024-04-26T00:35:29.015703+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22005,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949518+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:28.949518+0000","last_peered":"2024-04-26T00:35:28.949518+0000","last_clean":"2024-04-26T00:35:28.949518+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:28.949518+0000","last_undegraded":"2024-04-26T00:35:28.949518+0000","last_fullsized":"2024-04-26T00:35:28.949518+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22070,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949569+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:28.949569+0000","last_peered":"2024-04-26T00:35:28.949569+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:28.949569+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21625,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949367+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:28.949367+0000","last_peered":"2024-04-26T00:35:28.949367+0000","last_clean":"2024-04-26T00:35:28.949367+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:28.949367+0000","last_undegraded":"2024-04-26T00:35:28.949367+0000","last_fullsized":"2024-04-26T00:35:28.949367+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":26677,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:45.013605+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:45.013605+0000","last_peered":"2024-04-26T00:35:45.013605+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:45.013605+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":22208,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949591+0000","last_change":"2024-04-26T00:26:01.839861+0000","last_active":"2024-04-26T00:35:28.949591+0000","last_peered":"2024-04-26T00:35:28.949591+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:28.949591+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3302,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3300}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.949490+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:28.949490+0000","last_peered":"2024-04-26T00:35:28.949490+0000","last_clean":"2024-04-26T00:35:28.949490+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:28.949490+0000","last_undegraded":"2024-04-26T00:35:28.949490+0000","last_fullsized":"2024-04-26T00:35:28.949490+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22042,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.949668+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:28.949668+0000","last_peered":"2024-04-26T00:35:28.949668+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:28.949668+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21820,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950922+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:28.950922+0000","last_peered":"2024-04-26T00:35:28.950922+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:28.950922+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22150,"reported_epoch":48,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:28.950027+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:28.950027+0000","last_peered":"2024-04-26T00:35:28.950027+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:28.950027+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17325,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25886,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176218112,"data_stored":176171143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487484,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884356,"statfs":{"total":96636764160,"available":96137580544,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":498724559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280650,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548636,"kb_used_data":57484,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823204,"statfs":{"total":96636764160,"available":96074960896,"internally_reserved":0,"allocated":58863616,"data_stored":58593196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084423,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484748,"kb_used_data":115712,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93887092,"statfs":{"total":96636764160,"available":96140382208,"internally_reserved":0,"allocated":118489088,"data_stored":118213331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117710848,"data_stored":117707235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:52.408 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:52.770 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:52.771 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:52.784 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-04-26T00:35:52.580487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26540,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520884,"kb_used_data":173648,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281594636,"statfs":{"total":289910292480,"available":288352907264,"internally_reserved":0,"allocated":177815552,"data_stored":176952130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1577,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001393"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.015873+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.015873+0000","last_peered":"2024-04-26T00:35:50.015873+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.015873+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":26677,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:45.013605+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:45.013605+0000","last_peered":"2024-04-26T00:35:45.013605+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:45.013605+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":23521,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:52.300738+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:35:52.300738+0000","last_peered":"2024-04-26T00:35:52.300738+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:52.300738+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016075+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016075+0000","last_peered":"2024-04-26T00:35:50.016075+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016075+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26540,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176218112,"data_stored":176171143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487484,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884356,"statfs":{"total":96636764160,"available":96137580544,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":498724559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280651,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084423,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484748,"kb_used_data":115712,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93887092,"statfs":{"total":96636764160,"available":96140382208,"internally_reserved":0,"allocated":118489088,"data_stored":118213331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117710848,"data_stored":117707235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:52.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:52.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22074, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.015873+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:50.015873+0000', 'last_peered': '2024-04-26T00:35:50.015873+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:50.015873+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:52.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-04-26T00:35:52.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "44'21067", 'reported_seq': 26677, 'reported_epoch': 48, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:45.013605+0000', 'last_change': '2024-04-26T00:35:27.654706+0000', 'last_active': '2024-04-26T00:35:45.013605+0000', 'last_peered': '2024-04-26T00:35:45.013605+0000', 'last_clean': '2024-04-26T00:25:59.323315+0000', 'last_became_active': '2024-04-26T00:26:01.833524+0000', 'last_became_peered': '2024-04-26T00:26:01.833524+0000', 'last_unstale': '2024-04-26T00:35:45.013605+0000', 'last_undegraded': '2024-04-26T00:26:01.406292+0000', 'last_fullsized': '2024-04-26T00:26:01.367226+0000', 'mapping_epoch': 35, 'log_start': "38'11000", 'ondisk_log_start': "38'11000", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'objects_scrubbed': 1024, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': True, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 966, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 4, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 768, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20693, 'num_write_kb': 41380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2699, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -766}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-04-26T00:35:52.785 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:52.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 23521, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:52.300738+0000', 'last_change': '2024-04-26T00:35:48.231309+0000', 'last_active': '2024-04-26T00:35:52.300738+0000', 'last_peered': '2024-04-26T00:35:52.300738+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:52.300738+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2648, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 976, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2648}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:52.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:52.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22046, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016075+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:50.016075+0000', 'last_peered': '2024-04-26T00:35:50.016075+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:50.016075+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:52.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:52.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21824, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016419+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:50.016419+0000', 'last_peered': '2024-04-26T00:35:50.016419+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:50.016419+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:52.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:52.786 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22154, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016206+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:50.016206+0000', 'last_peered': '2024-04-26T00:35:50.016206+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:50.016206+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:52.786 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:53.260 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:53.260 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16671/6 objects degraded (277850.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":699,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":2}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1557385216,"bytes_avail":288352907264,"bytes_total":289910292480,"degraded_objects":16671,"degraded_total":6,"degraded_ratio":2778.5,"recovering_objects_per_sec":197,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1577,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:53.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:53.274 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:53.621 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:53.622 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:53.636 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":393,"stamp":"2024-04-26T00:35:52.580487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26540,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520884,"kb_used_data":173648,"kb_used_omap":12,"kb_used_meta":1347187,"kb_avail":281594636,"statfs":{"total":289910292480,"available":288352907264,"internally_reserved":0,"allocated":177815552,"data_stored":176952130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1379519596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1577,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1577,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001393"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28570,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952536+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:28.952536+0000","last_peered":"2024-04-26T00:35:28.952536+0000","last_clean":"2024-04-26T00:35:28.952536+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:28.952536+0000","last_undegraded":"2024-04-26T00:35:28.952536+0000","last_fullsized":"2024-04-26T00:35:28.952536+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.015873+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.015873+0000","last_peered":"2024-04-26T00:35:50.015873+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.015873+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28318,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952339+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:28.952339+0000","last_peered":"2024-04-26T00:35:28.952339+0000","last_clean":"2024-04-26T00:35:28.952339+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:28.952339+0000","last_undegraded":"2024-04-26T00:35:28.952339+0000","last_fullsized":"2024-04-26T00:35:28.952339+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952317+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:28.952317+0000","last_peered":"2024-04-26T00:35:28.952317+0000","last_clean":"2024-04-26T00:35:28.952317+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:28.952317+0000","last_undegraded":"2024-04-26T00:35:28.952317+0000","last_fullsized":"2024-04-26T00:35:28.952317+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":26677,"reported_epoch":48,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:45.013605+0000","last_change":"2024-04-26T00:35:27.654706+0000","last_active":"2024-04-26T00:35:45.013605+0000","last_peered":"2024-04-26T00:35:45.013605+0000","last_clean":"2024-04-26T00:25:59.323315+0000","last_became_active":"2024-04-26T00:26:01.833524+0000","last_became_peered":"2024-04-26T00:26:01.833524+0000","last_unstale":"2024-04-26T00:35:45.013605+0000","last_undegraded":"2024-04-26T00:26:01.406292+0000","last_fullsized":"2024-04-26T00:26:01.367226+0000","mapping_epoch":35,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2699,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-766}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":23521,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:52.300738+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:35:52.300738+0000","last_peered":"2024-04-26T00:35:52.300738+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:52.300738+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952400+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:28.952400+0000","last_peered":"2024-04-26T00:35:28.952400+0000","last_clean":"2024-04-26T00:35:28.952400+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:28.952400+0000","last_undegraded":"2024-04-26T00:35:28.952400+0000","last_fullsized":"2024-04-26T00:35:28.952400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016075+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016075+0000","last_peered":"2024-04-26T00:35:50.016075+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016075+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952632+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:28.952632+0000","last_peered":"2024-04-26T00:35:28.952632+0000","last_clean":"2024-04-26T00:35:28.952632+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:28.952632+0000","last_undegraded":"2024-04-26T00:35:28.952632+0000","last_fullsized":"2024-04-26T00:35:28.952632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952531+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:28.952531+0000","last_peered":"2024-04-26T00:35:28.952531+0000","last_clean":"2024-04-26T00:35:28.952531+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:28.952531+0000","last_undegraded":"2024-04-26T00:35:28.952531+0000","last_fullsized":"2024-04-26T00:35:28.952531+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":83,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952421+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:28.952421+0000","last_peered":"2024-04-26T00:35:28.952421+0000","last_clean":"2024-04-26T00:35:28.952421+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:28.952421+0000","last_undegraded":"2024-04-26T00:35:28.952421+0000","last_fullsized":"2024-04-26T00:35:28.952421+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28746,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-04-26T00:35:28.952533+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:28.952533+0000","last_peered":"2024-04-26T00:35:28.952533+0000","last_clean":"2024-04-26T00:35:28.952533+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:28.952533+0000","last_undegraded":"2024-04-26T00:35:28.952533+0000","last_fullsized":"2024-04-26T00:35:28.952533+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26540,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176218112,"data_stored":176171143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487484,"kb_used_data":436,"kb_used_omap":4,"kb_used_meta":487035,"kb_avail":93884356,"statfs":{"total":96636764160,"available":96137580544,"internally_reserved":0,"allocated":446464,"data_stored":139914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":498724559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280651,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084423,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484748,"kb_used_data":115712,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93887092,"statfs":{"total":96636764160,"available":96140382208,"internally_reserved":0,"allocated":118489088,"data_stored":118213331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":117710848,"data_stored":117707235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:56.638 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:56.942 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:56.943 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:56.954 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-04-26T00:35:56.581145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27308,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522896,"kb_used_data":171504,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281592624,"statfs":{"total":289910292480,"available":288350846976,"internally_reserved":0,"allocated":175620096,"data_stored":174747847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1422,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001417"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.015873+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.015873+0000","last_peered":"2024-04-26T00:35:50.015873+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.015873+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007712+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007712+0000","last_peered":"2024-04-26T00:35:50.007712+0000","last_clean":"2024-04-26T00:35:50.007712+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007712+0000","last_undegraded":"2024-04-26T00:35:50.007712+0000","last_fullsized":"2024-04-26T00:35:50.007712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":23521,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:52.300738+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:35:52.300738+0000","last_peered":"2024-04-26T00:35:52.300738+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:52.300738+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016075+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016075+0000","last_peered":"2024-04-26T00:35:50.016075+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016075+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28750,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007955+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007955+0000","last_peered":"2024-04-26T00:35:50.007955+0000","last_clean":"2024-04-26T00:35:50.007955+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007955+0000","last_undegraded":"2024-04-26T00:35:50.007955+0000","last_fullsized":"2024-04-26T00:35:50.007955+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27308,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174366720,"data_stored":174326977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491652,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880188,"statfs":{"total":96636764160,"available":96133312512,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280651,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084424,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482592,"kb_used_data":113560,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889248,"statfs":{"total":96636764160,"available":96142589952,"internally_reserved":0,"allocated":116285440,"data_stored":116004325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115867648,"data_stored":115864035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:56.956 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:57.284 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:57.284 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:57.296 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-04-26T00:35:56.581145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27308,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522896,"kb_used_data":171504,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281592624,"statfs":{"total":289910292480,"available":288350846976,"internally_reserved":0,"allocated":175620096,"data_stored":174747847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1422,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001417"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.015873+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.015873+0000","last_peered":"2024-04-26T00:35:50.015873+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.015873+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007712+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007712+0000","last_peered":"2024-04-26T00:35:50.007712+0000","last_clean":"2024-04-26T00:35:50.007712+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007712+0000","last_undegraded":"2024-04-26T00:35:50.007712+0000","last_fullsized":"2024-04-26T00:35:50.007712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":23521,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:52.300738+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:35:52.300738+0000","last_peered":"2024-04-26T00:35:52.300738+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:52.300738+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016075+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016075+0000","last_peered":"2024-04-26T00:35:50.016075+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016075+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28750,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007955+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007955+0000","last_peered":"2024-04-26T00:35:50.007955+0000","last_clean":"2024-04-26T00:35:50.007955+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007955+0000","last_undegraded":"2024-04-26T00:35:50.007955+0000","last_fullsized":"2024-04-26T00:35:50.007955+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27308,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174366720,"data_stored":174326977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491652,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880188,"statfs":{"total":96636764160,"available":96133312512,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280651,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084424,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482592,"kb_used_data":113560,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889248,"statfs":{"total":96636764160,"available":96142589952,"internally_reserved":0,"allocated":116285440,"data_stored":116004325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115867648,"data_stored":115864035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:35:57.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:35:57.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22074, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.015873+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:50.015873+0000', 'last_peered': '2024-04-26T00:35:50.015873+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:50.015873+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:57.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:35:57.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 23521, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:52.300738+0000', 'last_change': '2024-04-26T00:35:48.231309+0000', 'last_active': '2024-04-26T00:35:52.300738+0000', 'last_peered': '2024-04-26T00:35:52.300738+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:52.300738+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2648, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 976, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2648}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:57.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:35:57.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22046, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016075+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:50.016075+0000', 'last_peered': '2024-04-26T00:35:50.016075+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:50.016075+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:57.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:35:57.298 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21824, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016419+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:50.016419+0000', 'last_peered': '2024-04-26T00:35:50.016419+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:50.016419+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:57.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:35:57.299 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22154, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016206+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:50.016206+0000', 'last_peered': '2024-04-26T00:35:50.016206+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:50.016206+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:35:57.299 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:35:57.679 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:57.679 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15903/6 objects degraded (265050.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1559445504,"bytes_avail":288350846976,"bytes_total":289910292480,"degraded_objects":15903,"degraded_total":6,"degraded_ratio":2650.5,"recovering_objects_per_sec":177,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1422,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:35:57.692 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:35:57.693 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:35:58.011 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:35:58.011 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:35:58.024 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":395,"stamp":"2024-04-26T00:35:56.581145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27308,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522896,"kb_used_data":171504,"kb_used_omap":12,"kb_used_meta":1351347,"kb_avail":281592624,"statfs":{"total":289910292480,"available":288350846976,"internally_reserved":0,"allocated":175620096,"data_stored":174747847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1383779436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1422,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1422,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001417"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.015873+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.015873+0000","last_peered":"2024-04-26T00:35:50.015873+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.015873+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007712+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007712+0000","last_peered":"2024-04-26T00:35:50.007712+0000","last_clean":"2024-04-26T00:35:50.007712+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007712+0000","last_undegraded":"2024-04-26T00:35:50.007712+0000","last_fullsized":"2024-04-26T00:35:50.007712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":23521,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:52.300738+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:35:52.300738+0000","last_peered":"2024-04-26T00:35:52.300738+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:52.300738+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2648,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":976,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2648}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016075+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016075+0000","last_peered":"2024-04-26T00:35:50.016075+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016075+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28750,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007955+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007955+0000","last_peered":"2024-04-26T00:35:50.007955+0000","last_clean":"2024-04-26T00:35:50.007955+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007955+0000","last_undegraded":"2024-04-26T00:35:50.007955+0000","last_fullsized":"2024-04-26T00:35:50.007955+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15903,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27308,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174366720,"data_stored":174326977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491652,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":491195,"kb_avail":93880188,"statfs":{"total":96636764160,"available":96133312512,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":502984399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280651,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084424,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482592,"kb_used_data":113560,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93889248,"statfs":{"total":96636764160,"available":96142589952,"internally_reserved":0,"allocated":116285440,"data_stored":116004325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":115867648,"data_stored":115864035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:01.026 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:01.361 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:01.362 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:01.375 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-04-26T00:36:00.581843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28087,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512776,"kb_used_data":169704,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281602744,"statfs":{"total":289910292480,"available":288361209856,"internally_reserved":0,"allocated":173776896,"data_stored":172904647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.575289"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22075,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.017005+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.017005+0000","last_peered":"2024-04-26T00:35:50.017005+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.017005+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007712+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007712+0000","last_peered":"2024-04-26T00:35:50.007712+0000","last_clean":"2024-04-26T00:35:50.007712+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007712+0000","last_undegraded":"2024-04-26T00:35:50.007712+0000","last_fullsized":"2024-04-26T00:35:50.007712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":25079,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:57.301126+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:35:57.301126+0000","last_peered":"2024-04-26T00:35:57.301126+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:57.301126+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1869,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1755,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1869}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22047,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016875+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016875+0000","last_peered":"2024-04-26T00:35:50.016875+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016875+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28750,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007955+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007955+0000","last_peered":"2024-04-26T00:35:50.007955+0000","last_clean":"2024-04-26T00:35:50.007955+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007955+0000","last_undegraded":"2024-04-26T00:35:50.007955+0000","last_fullsized":"2024-04-26T00:35:50.007955+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28087,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172523520,"data_stored":172483777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280652,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084425,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480792,"kb_used_data":111760,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93891048,"statfs":{"total":96636764160,"available":96144433152,"internally_reserved":0,"allocated":114442240,"data_stored":114161125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114024448,"data_stored":114020835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:01.378 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:01.691 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:01.692 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:01.704 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-04-26T00:36:00.581843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28087,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512776,"kb_used_data":169704,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281602744,"statfs":{"total":289910292480,"available":288361209856,"internally_reserved":0,"allocated":173776896,"data_stored":172904647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.575289"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22075,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.017005+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.017005+0000","last_peered":"2024-04-26T00:35:50.017005+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.017005+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007712+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007712+0000","last_peered":"2024-04-26T00:35:50.007712+0000","last_clean":"2024-04-26T00:35:50.007712+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007712+0000","last_undegraded":"2024-04-26T00:35:50.007712+0000","last_fullsized":"2024-04-26T00:35:50.007712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":25079,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:57.301126+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:35:57.301126+0000","last_peered":"2024-04-26T00:35:57.301126+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:57.301126+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1869,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1755,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1869}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22047,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016875+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016875+0000","last_peered":"2024-04-26T00:35:50.016875+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016875+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28750,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007955+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007955+0000","last_peered":"2024-04-26T00:35:50.007955+0000","last_clean":"2024-04-26T00:35:50.007955+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007955+0000","last_undegraded":"2024-04-26T00:35:50.007955+0000","last_fullsized":"2024-04-26T00:35:50.007955+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28087,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172523520,"data_stored":172483777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280652,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084425,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480792,"kb_used_data":111760,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93891048,"statfs":{"total":96636764160,"available":96144433152,"internally_reserved":0,"allocated":114442240,"data_stored":114161125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114024448,"data_stored":114020835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:01.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:01.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22075, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.017005+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:50.017005+0000', 'last_peered': '2024-04-26T00:35:50.017005+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:50.017005+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:01.705 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:36:01.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 25079, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:57.301126+0000', 'last_change': '2024-04-26T00:35:48.231309+0000', 'last_active': '2024-04-26T00:35:57.301126+0000', 'last_peered': '2024-04-26T00:35:57.301126+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:35:57.301126+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1869, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1755, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1869}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:01.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:01.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22047, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016875+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:50.016875+0000', 'last_peered': '2024-04-26T00:35:50.016875+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:50.016875+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:01.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:01.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21824, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016419+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:50.016419+0000', 'last_peered': '2024-04-26T00:35:50.016419+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:50.016419+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:01.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:36:01.706 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22154, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016206+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:50.016206+0000', 'last_peered': '2024-04-26T00:35:50.016206+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:50.016206+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:01.706 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:02.106 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:02.106 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15124/6 objects degraded (252066.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":708,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1549082624,"bytes_avail":288361209856,"bytes_total":289910292480,"degraded_objects":15124,"degraded_total":6,"degraded_ratio":2520.6666666666665,"recovering_objects_per_sec":208,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2201,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:02.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:02.120 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:02.414 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:02.415 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:02.427 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2024-04-26T00:36:00.581843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28087,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1512776,"kb_used_data":169704,"kb_used_omap":12,"kb_used_meta":1343027,"kb_avail":281602744,"statfs":{"total":289910292480,"available":288361209856,"internally_reserved":0,"allocated":173776896,"data_stored":172904647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1375259756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.575289"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22075,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.017005+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.017005+0000","last_peered":"2024-04-26T00:35:50.017005+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.017005+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007712+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007712+0000","last_peered":"2024-04-26T00:35:50.007712+0000","last_clean":"2024-04-26T00:35:50.007712+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007712+0000","last_undegraded":"2024-04-26T00:35:50.007712+0000","last_fullsized":"2024-04-26T00:35:50.007712+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":25079,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:57.301126+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:35:57.301126+0000","last_peered":"2024-04-26T00:35:57.301126+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:35:57.301126+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1869,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1755,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1869}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22047,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016875+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016875+0000","last_peered":"2024-04-26T00:35:50.016875+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016875+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28750,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007955+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007955+0000","last_peered":"2024-04-26T00:35:50.007955+0000","last_clean":"2024-04-26T00:35:50.007955+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007955+0000","last_undegraded":"2024-04-26T00:35:50.007955+0000","last_fullsized":"2024-04-26T00:35:50.007955+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15124,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28087,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172523520,"data_stored":172483777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483332,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":482875,"kb_avail":93888508,"statfs":{"total":96636764160,"available":96141832192,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":494464719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280652,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084425,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480792,"kb_used_data":111760,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93891048,"statfs":{"total":96636764160,"available":96144433152,"internally_reserved":0,"allocated":114442240,"data_stored":114161125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114024448,"data_stored":114020835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:05.429 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:05.733 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:05.734 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:05.746 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-04-26T00:36:04.582632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28947,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1539144,"kb_used_data":169704,"kb_used_omap":12,"kb_used_meta":1369395,"kb_avail":281576376,"statfs":{"total":289910292480,"available":288334209024,"internally_reserved":0,"allocated":173776896,"data_stored":172904647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1402260588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002144"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22075,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.017005+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.017005+0000","last_peered":"2024-04-26T00:35:50.017005+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.017005+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007760+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007760+0000","last_peered":"2024-04-26T00:35:50.007760+0000","last_clean":"2024-04-26T00:35:50.007760+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007760+0000","last_undegraded":"2024-04-26T00:35:50.007760+0000","last_fullsized":"2024-04-26T00:35:50.007760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":26799,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:02.303617+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:36:02.303617+0000","last_peered":"2024-04-26T00:36:02.303617+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:36:02.303617+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1009}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22047,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016875+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016875+0000","last_peered":"2024-04-26T00:35:50.016875+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016875+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28751,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007996+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007996+0000","last_peered":"2024-04-26T00:35:50.007996+0000","last_clean":"2024-04-26T00:35:50.007996+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007996+0000","last_undegraded":"2024-04-26T00:35:50.007996+0000","last_fullsized":"2024-04-26T00:35:50.007996+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28947,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172523520,"data_stored":172483777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509700,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862140,"statfs":{"total":96636764160,"available":96114831360,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280653,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084425,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480792,"kb_used_data":111760,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93891048,"statfs":{"total":96636764160,"available":96144433152,"internally_reserved":0,"allocated":114442240,"data_stored":114161125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114024448,"data_stored":114020835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:05.748 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:06.049 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:06.049 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:06.062 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-04-26T00:36:04.582632+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28947,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1539144,"kb_used_data":169704,"kb_used_omap":12,"kb_used_meta":1369395,"kb_avail":281576376,"statfs":{"total":289910292480,"available":288334209024,"internally_reserved":0,"allocated":173776896,"data_stored":172904647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1402260588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002144"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22075,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.017005+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.017005+0000","last_peered":"2024-04-26T00:35:50.017005+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.017005+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007760+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007760+0000","last_peered":"2024-04-26T00:35:50.007760+0000","last_clean":"2024-04-26T00:35:50.007760+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007760+0000","last_undegraded":"2024-04-26T00:35:50.007760+0000","last_fullsized":"2024-04-26T00:35:50.007760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":26799,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:02.303617+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:36:02.303617+0000","last_peered":"2024-04-26T00:36:02.303617+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:36:02.303617+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1009}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22047,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016875+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016875+0000","last_peered":"2024-04-26T00:35:50.016875+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016875+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28751,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007996+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007996+0000","last_peered":"2024-04-26T00:35:50.007996+0000","last_clean":"2024-04-26T00:35:50.007996+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007996+0000","last_undegraded":"2024-04-26T00:35:50.007996+0000","last_fullsized":"2024-04-26T00:35:50.007996+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28947,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":172523520,"data_stored":172483777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509700,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862140,"statfs":{"total":96636764160,"available":96114831360,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280653,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084425,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480792,"kb_used_data":111760,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93891048,"statfs":{"total":96636764160,"available":96144433152,"internally_reserved":0,"allocated":114442240,"data_stored":114161125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114024448,"data_stored":114020835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:06.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:06.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22075, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.017005+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:35:50.017005+0000', 'last_peered': '2024-04-26T00:35:50.017005+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:35:50.017005+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:06.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:36:06.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 26799, 'reported_epoch': 50, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:02.303617+0000', 'last_change': '2024-04-26T00:35:48.231309+0000', 'last_active': '2024-04-26T00:36:02.303617+0000', 'last_peered': '2024-04-26T00:36:02.303617+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:36:02.303617+0000', 'last_undegraded': '2024-04-26T00:26:01.453652+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1009, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2615, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1009}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:06.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:06.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22047, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016875+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:35:50.016875+0000', 'last_peered': '2024-04-26T00:35:50.016875+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:35:50.016875+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:06.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:06.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21824, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016419+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:35:50.016419+0000', 'last_peered': '2024-04-26T00:35:50.016419+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:35:50.016419+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:06.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:36:06.064 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22154, 'reported_epoch': 50, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:35:50.016206+0000', 'last_change': '2024-04-26T00:26:01.844300+0000', 'last_active': '2024-04-26T00:35:50.016206+0000', 'last_peered': '2024-04-26T00:35:50.016206+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:35:50.016206+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3365, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 292, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:06.065 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:06.435 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:06.435 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14264/6 objects degraded (237733.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":712,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":50,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1576083456,"bytes_avail":288334209024,"bytes_total":289910292480,"degraded_objects":14264,"degraded_total":6,"degraded_ratio":2377.3333333333335,"recovering_objects_per_sec":200,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2407,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:06.448 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:06.448 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:06.755 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:06.755 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:06.769 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":400,"stamp":"2024-04-26T00:36:06.582982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28947,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1537344,"kb_used_data":167904,"kb_used_omap":12,"kb_used_meta":1369395,"kb_avail":281578176,"statfs":{"total":289910292480,"available":288336052224,"internally_reserved":0,"allocated":171933696,"data_stored":171061447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1402260588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1639,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002152"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28620,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.017243+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:35:50.017243+0000","last_peered":"2024-04-26T00:35:50.017243+0000","last_clean":"2024-04-26T00:35:50.017243+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:35:50.017243+0000","last_undegraded":"2024-04-26T00:35:50.017243+0000","last_fullsized":"2024-04-26T00:35:50.017243+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28574,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008052+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:35:50.008052+0000","last_peered":"2024-04-26T00:35:50.008052+0000","last_clean":"2024-04-26T00:35:50.008052+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:35:50.008052+0000","last_undegraded":"2024-04-26T00:35:50.008052+0000","last_fullsized":"2024-04-26T00:35:50.008052+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22009,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.016037+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:35:50.016037+0000","last_peered":"2024-04-26T00:35:50.016037+0000","last_clean":"2024-04-26T00:35:50.016037+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:35:50.016037+0000","last_undegraded":"2024-04-26T00:35:50.016037+0000","last_fullsized":"2024-04-26T00:35:50.016037+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22075,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.017005+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:35:50.017005+0000","last_peered":"2024-04-26T00:35:50.017005+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:35:50.017005+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28322,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007894+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:35:50.007894+0000","last_peered":"2024-04-26T00:35:50.007894+0000","last_clean":"2024-04-26T00:35:50.007894+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:35:50.007894+0000","last_undegraded":"2024-04-26T00:35:50.007894+0000","last_fullsized":"2024-04-26T00:35:50.007894+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21629,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015692+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:35:50.015692+0000","last_peered":"2024-04-26T00:35:50.015692+0000","last_clean":"2024-04-26T00:35:50.015692+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:35:50.015692+0000","last_undegraded":"2024-04-26T00:35:50.015692+0000","last_fullsized":"2024-04-26T00:35:50.015692+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007760+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:35:50.007760+0000","last_peered":"2024-04-26T00:35:50.007760+0000","last_clean":"2024-04-26T00:35:50.007760+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:35:50.007760+0000","last_undegraded":"2024-04-26T00:35:50.007760+0000","last_fullsized":"2024-04-26T00:35:50.007760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21645,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.103634+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:35:50.103634+0000","last_peered":"2024-04-26T00:35:50.103634+0000","last_clean":"2024-04-26T00:35:50.103634+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:35:50.103634+0000","last_undegraded":"2024-04-26T00:35:50.103634+0000","last_fullsized":"2024-04-26T00:35:50.103634+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:35:55.186594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":26799,"reported_epoch":50,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:02.303617+0000","last_change":"2024-04-26T00:35:48.231309+0000","last_active":"2024-04-26T00:36:02.303617+0000","last_peered":"2024-04-26T00:36:02.303617+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:36:02.303617+0000","last_undegraded":"2024-04-26T00:26:01.453652+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2615,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1009}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.015841+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:35:50.015841+0000","last_peered":"2024-04-26T00:35:50.015841+0000","last_clean":"2024-04-26T00:35:50.015841+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:35:50.015841+0000","last_undegraded":"2024-04-26T00:35:50.015841+0000","last_fullsized":"2024-04-26T00:35:50.015841+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007714+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:35:50.007714+0000","last_peered":"2024-04-26T00:35:50.007714+0000","last_clean":"2024-04-26T00:35:50.007714+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:35:50.007714+0000","last_undegraded":"2024-04-26T00:35:50.007714+0000","last_fullsized":"2024-04-26T00:35:50.007714+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22047,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016875+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:35:50.016875+0000","last_peered":"2024-04-26T00:35:50.016875+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:35:50.016875+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008042+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:35:50.008042+0000","last_peered":"2024-04-26T00:35:50.008042+0000","last_clean":"2024-04-26T00:35:50.008042+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:35:50.008042+0000","last_undegraded":"2024-04-26T00:35:50.008042+0000","last_fullsized":"2024-04-26T00:35:50.008042+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21824,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016419+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:35:50.016419+0000","last_peered":"2024-04-26T00:35:50.016419+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:35:50.016419+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22154,"reported_epoch":50,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:35:50.016206+0000","last_change":"2024-04-26T00:26:01.844300+0000","last_active":"2024-04-26T00:35:50.016206+0000","last_peered":"2024-04-26T00:35:50.016206+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:35:50.016206+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3363}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":83,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007998+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:35:50.007998+0000","last_peered":"2024-04-26T00:35:50.007998+0000","last_clean":"2024-04-26T00:35:50.007998+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:35:50.007998+0000","last_undegraded":"2024-04-26T00:35:50.007998+0000","last_fullsized":"2024-04-26T00:35:50.007998+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":87,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008146+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:35:50.008146+0000","last_peered":"2024-04-26T00:35:50.008146+0000","last_clean":"2024-04-26T00:35:50.008146+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:35:50.008146+0000","last_undegraded":"2024-04-26T00:35:50.008146+0000","last_fullsized":"2024-04-26T00:35:50.008146+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28751,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007996+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:35:50.007996+0000","last_peered":"2024-04-26T00:35:50.007996+0000","last_clean":"2024-04-26T00:35:50.007996+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:35:50.007996+0000","last_undegraded":"2024-04-26T00:35:50.007996+0000","last_fullsized":"2024-04-26T00:35:50.007996+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28947,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170680320,"data_stored":170640577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509700,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862140,"statfs":{"total":96636764160,"available":96114831360,"internally_reserved":0,"allocated":454656,"data_stored":144637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280653,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084426,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478992,"kb_used_data":109960,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93892848,"statfs":{"total":96636764160,"available":96146276352,"internally_reserved":0,"allocated":112599040,"data_stored":112317925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112181248,"data_stored":112177635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:09.771 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:10.160 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:10.160 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:10.174 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-04-26T00:36:08.583295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30042,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529028,"kb_used_data":167904,"kb_used_omap":12,"kb_used_meta":1361075,"kb_avail":281586492,"statfs":{"total":289910292480,"available":288344567808,"internally_reserved":0,"allocated":171933696,"data_stored":171063311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1393740908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001451"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28622,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.060304+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:07.060304+0000","last_peered":"2024-04-26T00:36:07.060304+0000","last_clean":"2024-04-26T00:36:07.060304+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:07.060304+0000","last_undegraded":"2024-04-26T00:36:07.060304+0000","last_fullsized":"2024-04-26T00:36:07.060304+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28576,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.069064+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:07.069064+0000","last_peered":"2024-04-26T00:36:07.069064+0000","last_clean":"2024-04-26T00:36:07.069064+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:07.069064+0000","last_undegraded":"2024-04-26T00:36:07.069064+0000","last_fullsized":"2024-04-26T00:36:07.069064+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22011,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059880+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:07.059880+0000","last_peered":"2024-04-26T00:36:07.059880+0000","last_clean":"2024-04-26T00:36:07.059880+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:07.059880+0000","last_undegraded":"2024-04-26T00:36:07.059880+0000","last_fullsized":"2024-04-26T00:36:07.059880+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22077,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.059667+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:07.059667+0000","last_peered":"2024-04-26T00:36:07.059667+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:07.059667+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28324,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068745+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:07.068745+0000","last_peered":"2024-04-26T00:36:07.068745+0000","last_clean":"2024-04-26T00:36:07.068745+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:07.068745+0000","last_undegraded":"2024-04-26T00:36:07.068745+0000","last_fullsized":"2024-04-26T00:36:07.068745+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21631,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059716+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:07.059716+0000","last_peered":"2024-04-26T00:36:07.059716+0000","last_clean":"2024-04-26T00:36:07.059716+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:07.059716+0000","last_undegraded":"2024-04-26T00:36:07.059716+0000","last_fullsized":"2024-04-26T00:36:07.059716+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068709+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:07.068709+0000","last_peered":"2024-04-26T00:36:07.068709+0000","last_clean":"2024-04-26T00:36:07.068709+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:07.068709+0000","last_undegraded":"2024-04-26T00:36:07.068709+0000","last_fullsized":"2024-04-26T00:36:07.068709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068868+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:07.068868+0000","last_peered":"2024-04-26T00:36:07.068868+0000","last_clean":"2024-04-26T00:36:07.068868+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:07.068868+0000","last_undegraded":"2024-04-26T00:36:07.068868+0000","last_fullsized":"2024-04-26T00:36:07.068868+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28819,"reported_epoch":50,"state":"peering","last_fresh":"2024-04-26T00:36:06.659601+0000","last_change":"2024-04-26T00:36:07.067380+0000","last_active":"2024-04-26T00:36:07.067380+0000","last_peered":"2024-04-26T00:36:06.659601+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:36:06.659601+0000","last_undegraded":"2024-04-26T00:36:06.659601+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059577+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:07.059577+0000","last_peered":"2024-04-26T00:36:07.059577+0000","last_clean":"2024-04-26T00:36:07.059577+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:07.059577+0000","last_undegraded":"2024-04-26T00:36:07.059577+0000","last_fullsized":"2024-04-26T00:36:07.059577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068822+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:07.068822+0000","last_peered":"2024-04-26T00:36:07.068822+0000","last_clean":"2024-04-26T00:36:07.068822+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:07.068822+0000","last_undegraded":"2024-04-26T00:36:07.068822+0000","last_fullsized":"2024-04-26T00:36:07.068822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22049,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.060010+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:07.060010+0000","last_peered":"2024-04-26T00:36:07.060010+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:07.060010+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068908+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:07.068908+0000","last_peered":"2024-04-26T00:36:07.068908+0000","last_clean":"2024-04-26T00:36:07.068908+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:07.068908+0000","last_undegraded":"2024-04-26T00:36:07.068908+0000","last_fullsized":"2024-04-26T00:36:07.068908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21826,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.060187+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:07.060187+0000","last_peered":"2024-04-26T00:36:07.060187+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:07.060187+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22329,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.302862+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:07.302862+0000","last_peered":"2024-04-26T00:36:07.302862+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:07.302862+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068922+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:07.068922+0000","last_peered":"2024-04-26T00:36:07.068922+0000","last_clean":"2024-04-26T00:36:07.068922+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:07.068922+0000","last_undegraded":"2024-04-26T00:36:07.068922+0000","last_fullsized":"2024-04-26T00:36:07.068922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068918+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:07.068918+0000","last_peered":"2024-04-26T00:36:07.068918+0000","last_clean":"2024-04-26T00:36:07.068918+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:07.068918+0000","last_undegraded":"2024-04-26T00:36:07.068918+0000","last_fullsized":"2024-04-26T00:36:07.068918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28753,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068867+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:07.068867+0000","last_peered":"2024-04-26T00:36:07.068867+0000","last_clean":"2024-04-26T00:36:07.068867+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:07.068867+0000","last_undegraded":"2024-04-26T00:36:07.068867+0000","last_fullsized":"2024-04-26T00:36:07.068867+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30042,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170672128,"data_stored":170639611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501384,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870456,"statfs":{"total":96636764160,"available":96123346944,"internally_reserved":0,"allocated":454656,"data_stored":146501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280654,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084426,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478992,"kb_used_data":109960,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93892848,"statfs":{"total":96636764160,"available":96146276352,"internally_reserved":0,"allocated":112599040,"data_stored":112317925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112181248,"data_stored":112177635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:10.175 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:10.567 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:10.567 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:10.580 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-04-26T00:36:08.583295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30042,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1529028,"kb_used_data":167904,"kb_used_omap":12,"kb_used_meta":1361075,"kb_avail":281586492,"statfs":{"total":289910292480,"available":288344567808,"internally_reserved":0,"allocated":171933696,"data_stored":171063311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1393740908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1955,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001451"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28622,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.060304+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:07.060304+0000","last_peered":"2024-04-26T00:36:07.060304+0000","last_clean":"2024-04-26T00:36:07.060304+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:07.060304+0000","last_undegraded":"2024-04-26T00:36:07.060304+0000","last_fullsized":"2024-04-26T00:36:07.060304+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21791,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008780+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:35:50.008780+0000","last_peered":"2024-04-26T00:35:50.008780+0000","last_clean":"2024-04-26T00:35:50.008780+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:35:50.008780+0000","last_undegraded":"2024-04-26T00:35:50.008780+0000","last_fullsized":"2024-04-26T00:35:50.008780+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28576,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.069064+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:07.069064+0000","last_peered":"2024-04-26T00:36:07.069064+0000","last_clean":"2024-04-26T00:36:07.069064+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:07.069064+0000","last_undegraded":"2024-04-26T00:36:07.069064+0000","last_fullsized":"2024-04-26T00:36:07.069064+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21163,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008515+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:35:50.008515+0000","last_peered":"2024-04-26T00:35:50.008515+0000","last_clean":"2024-04-26T00:35:50.008515+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:35:50.008515+0000","last_undegraded":"2024-04-26T00:35:50.008515+0000","last_fullsized":"2024-04-26T00:35:50.008515+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22011,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059880+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:07.059880+0000","last_peered":"2024-04-26T00:36:07.059880+0000","last_clean":"2024-04-26T00:36:07.059880+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:07.059880+0000","last_undegraded":"2024-04-26T00:36:07.059880+0000","last_fullsized":"2024-04-26T00:36:07.059880+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22077,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.059667+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:07.059667+0000","last_peered":"2024-04-26T00:36:07.059667+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:07.059667+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28324,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068745+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:07.068745+0000","last_peered":"2024-04-26T00:36:07.068745+0000","last_clean":"2024-04-26T00:36:07.068745+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:07.068745+0000","last_undegraded":"2024-04-26T00:36:07.068745+0000","last_fullsized":"2024-04-26T00:36:07.068745+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21631,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059716+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:07.059716+0000","last_peered":"2024-04-26T00:36:07.059716+0000","last_clean":"2024-04-26T00:36:07.059716+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:07.059716+0000","last_undegraded":"2024-04-26T00:36:07.059716+0000","last_fullsized":"2024-04-26T00:36:07.059716+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068709+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:07.068709+0000","last_peered":"2024-04-26T00:36:07.068709+0000","last_clean":"2024-04-26T00:36:07.068709+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:07.068709+0000","last_undegraded":"2024-04-26T00:36:07.068709+0000","last_fullsized":"2024-04-26T00:36:07.068709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068868+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:07.068868+0000","last_peered":"2024-04-26T00:36:07.068868+0000","last_clean":"2024-04-26T00:36:07.068868+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:07.068868+0000","last_undegraded":"2024-04-26T00:36:07.068868+0000","last_fullsized":"2024-04-26T00:36:07.068868+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008274+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:35:50.008274+0000","last_peered":"2024-04-26T00:35:50.008274+0000","last_clean":"2024-04-26T00:35:50.008274+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:35:50.008274+0000","last_undegraded":"2024-04-26T00:35:50.008274+0000","last_fullsized":"2024-04-26T00:35:50.008274+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28819,"reported_epoch":50,"state":"peering","last_fresh":"2024-04-26T00:36:06.659601+0000","last_change":"2024-04-26T00:36:07.067380+0000","last_active":"2024-04-26T00:36:07.067380+0000","last_peered":"2024-04-26T00:36:06.659601+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:36:06.659601+0000","last_undegraded":"2024-04-26T00:36:06.659601+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059577+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:07.059577+0000","last_peered":"2024-04-26T00:36:07.059577+0000","last_clean":"2024-04-26T00:36:07.059577+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:07.059577+0000","last_undegraded":"2024-04-26T00:36:07.059577+0000","last_fullsized":"2024-04-26T00:36:07.059577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28902,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008371+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:35:50.008371+0000","last_peered":"2024-04-26T00:35:50.008371+0000","last_clean":"2024-04-26T00:35:50.008371+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:35:50.008371+0000","last_undegraded":"2024-04-26T00:35:50.008371+0000","last_fullsized":"2024-04-26T00:35:50.008371+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068822+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:07.068822+0000","last_peered":"2024-04-26T00:36:07.068822+0000","last_clean":"2024-04-26T00:36:07.068822+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:07.068822+0000","last_undegraded":"2024-04-26T00:36:07.068822+0000","last_fullsized":"2024-04-26T00:36:07.068822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28336,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.007603+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:35:50.007603+0000","last_peered":"2024-04-26T00:35:50.007603+0000","last_clean":"2024-04-26T00:35:50.007603+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:35:50.007603+0000","last_undegraded":"2024-04-26T00:35:50.007603+0000","last_fullsized":"2024-04-26T00:35:50.007603+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22049,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.060010+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:07.060010+0000","last_peered":"2024-04-26T00:36:07.060010+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:07.060010+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068908+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:07.068908+0000","last_peered":"2024-04-26T00:36:07.068908+0000","last_clean":"2024-04-26T00:36:07.068908+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:07.068908+0000","last_undegraded":"2024-04-26T00:36:07.068908+0000","last_fullsized":"2024-04-26T00:36:07.068908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21826,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.060187+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:07.060187+0000","last_peered":"2024-04-26T00:36:07.060187+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:07.060187+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":67,"reported_epoch":50,"state":"active+clean","last_fresh":"2024-04-26T00:35:50.008685+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:35:50.008685+0000","last_peered":"2024-04-26T00:35:50.008685+0000","last_clean":"2024-04-26T00:35:50.008685+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:35:50.008685+0000","last_undegraded":"2024-04-26T00:35:50.008685+0000","last_fullsized":"2024-04-26T00:35:50.008685+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22329,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.302862+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:07.302862+0000","last_peered":"2024-04-26T00:36:07.302862+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:07.302862+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068922+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:07.068922+0000","last_peered":"2024-04-26T00:36:07.068922+0000","last_clean":"2024-04-26T00:36:07.068922+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:07.068922+0000","last_undegraded":"2024-04-26T00:36:07.068922+0000","last_fullsized":"2024-04-26T00:36:07.068922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068918+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:07.068918+0000","last_peered":"2024-04-26T00:36:07.068918+0000","last_clean":"2024-04-26T00:36:07.068918+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:07.068918+0000","last_undegraded":"2024-04-26T00:36:07.068918+0000","last_fullsized":"2024-04-26T00:36:07.068918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28753,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068867+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:07.068867+0000","last_peered":"2024-04-26T00:36:07.068867+0000","last_clean":"2024-04-26T00:36:07.068867+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:07.068867+0000","last_undegraded":"2024-04-26T00:36:07.068867+0000","last_fullsized":"2024-04-26T00:36:07.068867+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30042,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":170672128,"data_stored":170639611,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501384,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870456,"statfs":{"total":96636764160,"available":96123346944,"internally_reserved":0,"allocated":454656,"data_stored":146501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280654,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084426,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478992,"kb_used_data":109960,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93892848,"statfs":{"total":96636764160,"available":96146276352,"internally_reserved":0,"allocated":112599040,"data_stored":112317925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112181248,"data_stored":112177635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:10.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:10.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22077, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:07.059667+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:07.059667+0000', 'last_peered': '2024-04-26T00:36:07.059667+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:07.059667+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:10.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-04-26T00:36:10.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "44'21424", 'reported_seq': 28819, 'reported_epoch': 50, 'state': 'peering', 'last_fresh': '2024-04-26T00:36:06.659601+0000', 'last_change': '2024-04-26T00:36:07.067380+0000', 'last_active': '2024-04-26T00:36:07.067380+0000', 'last_peered': '2024-04-26T00:36:06.659601+0000', 'last_clean': '2024-04-26T00:25:59.319325+0000', 'last_became_active': '2024-04-26T00:26:01.833223+0000', 'last_became_peered': '2024-04-26T00:26:01.833223+0000', 'last_unstale': '2024-04-26T00:36:06.659601+0000', 'last_undegraded': '2024-04-26T00:36:06.659601+0000', 'last_fullsized': '2024-04-26T00:26:01.361429+0000', 'mapping_epoch': 51, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3241", 'last_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:51.982004+0000', 'objects_scrubbed': 3211, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 965, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18205, 'num_write_kb': 29988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3624, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:10.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:10.581 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22049, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:07.060010+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:36:07.060010+0000', 'last_peered': '2024-04-26T00:36:07.060010+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:07.060010+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:10.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:10.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21826, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:07.060187+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:36:07.060187+0000', 'last_peered': '2024-04-26T00:36:07.060187+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:36:07.060187+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:10.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:36:10.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 22329, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:07.302862+0000', 'last_change': '2024-04-26T00:36:06.659921+0000', 'last_active': '2024-04-26T00:36:07.302862+0000', 'last_peered': '2024-04-26T00:36:07.302862+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:36:07.302862+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3279, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 378, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3279}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:10.582 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:10.991 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:10.992 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13169/6 objects degraded (219483.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":716,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1565724672,"bytes_avail":288344567808,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":13169,"degraded_total":6,"degraded_ratio":2194.8333333333335,"recovering_objects_per_sec":244,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1955,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:11.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:11.006 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:11.362 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:11.362 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:11.375 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":404,"stamp":"2024-04-26T00:36:10.583579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30042,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1527016,"kb_used_data":165884,"kb_used_omap":12,"kb_used_meta":1361075,"kb_avail":281588504,"statfs":{"total":289910292480,"available":288346628096,"internally_reserved":0,"allocated":169865216,"data_stored":168984086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1393740908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1095,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1095,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001424"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28622,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.060304+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:07.060304+0000","last_peered":"2024-04-26T00:36:07.060304+0000","last_clean":"2024-04-26T00:36:07.060304+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:07.060304+0000","last_undegraded":"2024-04-26T00:36:07.060304+0000","last_fullsized":"2024-04-26T00:36:07.060304+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078754+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078754+0000","last_peered":"2024-04-26T00:36:08.078754+0000","last_clean":"2024-04-26T00:36:08.078754+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078754+0000","last_undegraded":"2024-04-26T00:36:08.078754+0000","last_fullsized":"2024-04-26T00:36:08.078754+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28576,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.069064+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:07.069064+0000","last_peered":"2024-04-26T00:36:07.069064+0000","last_clean":"2024-04-26T00:36:07.069064+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:07.069064+0000","last_undegraded":"2024-04-26T00:36:07.069064+0000","last_fullsized":"2024-04-26T00:36:07.069064+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079408+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079408+0000","last_peered":"2024-04-26T00:36:08.079408+0000","last_clean":"2024-04-26T00:36:08.079408+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079408+0000","last_undegraded":"2024-04-26T00:36:08.079408+0000","last_fullsized":"2024-04-26T00:36:08.079408+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22011,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059880+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:07.059880+0000","last_peered":"2024-04-26T00:36:07.059880+0000","last_clean":"2024-04-26T00:36:07.059880+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:07.059880+0000","last_undegraded":"2024-04-26T00:36:07.059880+0000","last_fullsized":"2024-04-26T00:36:07.059880+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22077,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.059667+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:07.059667+0000","last_peered":"2024-04-26T00:36:07.059667+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:07.059667+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28324,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068745+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:07.068745+0000","last_peered":"2024-04-26T00:36:07.068745+0000","last_clean":"2024-04-26T00:36:07.068745+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:07.068745+0000","last_undegraded":"2024-04-26T00:36:07.068745+0000","last_fullsized":"2024-04-26T00:36:07.068745+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21631,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059716+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:07.059716+0000","last_peered":"2024-04-26T00:36:07.059716+0000","last_clean":"2024-04-26T00:36:07.059716+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:07.059716+0000","last_undegraded":"2024-04-26T00:36:07.059716+0000","last_fullsized":"2024-04-26T00:36:07.059716+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068709+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:07.068709+0000","last_peered":"2024-04-26T00:36:07.068709+0000","last_clean":"2024-04-26T00:36:07.068709+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:07.068709+0000","last_undegraded":"2024-04-26T00:36:07.068709+0000","last_fullsized":"2024-04-26T00:36:07.068709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21647,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068868+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:07.068868+0000","last_peered":"2024-04-26T00:36:07.068868+0000","last_clean":"2024-04-26T00:36:07.068868+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:07.068868+0000","last_undegraded":"2024-04-26T00:36:07.068868+0000","last_fullsized":"2024-04-26T00:36:07.068868+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28819,"reported_epoch":50,"state":"peering","last_fresh":"2024-04-26T00:36:06.659601+0000","last_change":"2024-04-26T00:36:07.067380+0000","last_active":"2024-04-26T00:36:07.067380+0000","last_peered":"2024-04-26T00:36:06.659601+0000","last_clean":"2024-04-26T00:25:59.319325+0000","last_became_active":"2024-04-26T00:26:01.833223+0000","last_became_peered":"2024-04-26T00:26:01.833223+0000","last_unstale":"2024-04-26T00:36:06.659601+0000","last_undegraded":"2024-04-26T00:36:06.659601+0000","last_fullsized":"2024-04-26T00:26:01.361429+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":96,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.059577+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:07.059577+0000","last_peered":"2024-04-26T00:36:07.059577+0000","last_clean":"2024-04-26T00:36:07.059577+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:07.059577+0000","last_undegraded":"2024-04-26T00:36:07.059577+0000","last_fullsized":"2024-04-26T00:36:07.059577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068822+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:07.068822+0000","last_peered":"2024-04-26T00:36:07.068822+0000","last_clean":"2024-04-26T00:36:07.068822+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:07.068822+0000","last_undegraded":"2024-04-26T00:36:07.068822+0000","last_fullsized":"2024-04-26T00:36:07.068822+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22049,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.060010+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:07.060010+0000","last_peered":"2024-04-26T00:36:07.060010+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:07.060010+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068908+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:07.068908+0000","last_peered":"2024-04-26T00:36:07.068908+0000","last_clean":"2024-04-26T00:36:07.068908+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:07.068908+0000","last_undegraded":"2024-04-26T00:36:07.068908+0000","last_fullsized":"2024-04-26T00:36:07.068908+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21826,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.060187+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:07.060187+0000","last_peered":"2024-04-26T00:36:07.060187+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:07.060187+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22329,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:07.302862+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:07.302862+0000","last_peered":"2024-04-26T00:36:07.302862+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:07.302862+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":378,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3279}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068922+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:07.068922+0000","last_peered":"2024-04-26T00:36:07.068922+0000","last_clean":"2024-04-26T00:36:07.068922+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:07.068922+0000","last_undegraded":"2024-04-26T00:36:07.068922+0000","last_fullsized":"2024-04-26T00:36:07.068922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068918+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:07.068918+0000","last_peered":"2024-04-26T00:36:07.068918+0000","last_clean":"2024-04-26T00:36:07.068918+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:07.068918+0000","last_undegraded":"2024-04-26T00:36:07.068918+0000","last_fullsized":"2024-04-26T00:36:07.068918+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28753,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-26T00:36:07.068867+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:07.068867+0000","last_peered":"2024-04-26T00:36:07.068867+0000","last_clean":"2024-04-26T00:36:07.068867+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:07.068867+0000","last_undegraded":"2024-04-26T00:36:07.068867+0000","last_fullsized":"2024-04-26T00:36:07.068867+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13169,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30042,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168587264,"data_stored":168554747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501384,"kb_used_data":444,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870456,"statfs":{"total":96636764160,"available":96123346944,"internally_reserved":0,"allocated":454656,"data_stored":146501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280654,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548652,"kb_used_data":57500,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823188,"statfs":{"total":96636764160,"available":96074944512,"internally_reserved":0,"allocated":58880000,"data_stored":58598885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084427,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476980,"kb_used_data":107940,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93894860,"statfs":{"total":96636764160,"available":96148336640,"internally_reserved":0,"allocated":110530560,"data_stored":110238700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110096384,"data_stored":110092771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:14.377 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:14.680 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:14.681 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:14.692 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-04-26T00:36:14.584246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31008,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535356,"kb_used_data":165908,"kb_used_omap":12,"kb_used_meta":1369395,"kb_avail":281580164,"statfs":{"total":289910292480,"available":288338087936,"internally_reserved":0,"allocated":169889792,"data_stored":168992534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1402260588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2061,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001264"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078754+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078754+0000","last_peered":"2024-04-26T00:36:08.078754+0000","last_clean":"2024-04-26T00:36:08.078754+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078754+0000","last_undegraded":"2024-04-26T00:36:08.078754+0000","last_fullsized":"2024-04-26T00:36:08.078754+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079408+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079408+0000","last_peered":"2024-04-26T00:36:08.079408+0000","last_clean":"2024-04-26T00:36:08.079408+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079408+0000","last_undegraded":"2024-04-26T00:36:08.079408+0000","last_fullsized":"2024-04-26T00:36:08.079408+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":24263,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:12.307313+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:12.307313+0000","last_peered":"2024-04-26T00:36:12.307313+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:12.307313+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31008,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168587264,"data_stored":168554747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509708,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862132,"statfs":{"total":96636764160,"available":96114823168,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280655,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084427,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476980,"kb_used_data":107940,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93894860,"statfs":{"total":96636764160,"available":96148336640,"internally_reserved":0,"allocated":110530560,"data_stored":110238700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110096384,"data_stored":110092771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:14.694 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:15.020 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:15.020 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:15.033 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-04-26T00:36:14.584246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31008,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535356,"kb_used_data":165908,"kb_used_omap":12,"kb_used_meta":1369395,"kb_avail":281580164,"statfs":{"total":289910292480,"available":288338087936,"internally_reserved":0,"allocated":169889792,"data_stored":168992534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1402260588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2061,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001264"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078754+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078754+0000","last_peered":"2024-04-26T00:36:08.078754+0000","last_clean":"2024-04-26T00:36:08.078754+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078754+0000","last_undegraded":"2024-04-26T00:36:08.078754+0000","last_fullsized":"2024-04-26T00:36:08.078754+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079408+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079408+0000","last_peered":"2024-04-26T00:36:08.079408+0000","last_clean":"2024-04-26T00:36:08.079408+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079408+0000","last_undegraded":"2024-04-26T00:36:08.079408+0000","last_fullsized":"2024-04-26T00:36:08.079408+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":24263,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:12.307313+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:12.307313+0000","last_peered":"2024-04-26T00:36:12.307313+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:12.307313+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31008,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168587264,"data_stored":168554747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509708,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862132,"statfs":{"total":96636764160,"available":96114823168,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280655,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084427,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476980,"kb_used_data":107940,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93894860,"statfs":{"total":96636764160,"available":96148336640,"internally_reserved":0,"allocated":110530560,"data_stored":110238700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110096384,"data_stored":110092771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:15.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:15.034 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22079, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080187+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:08.080187+0000', 'last_peered': '2024-04-26T00:36:08.080187+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:08.080187+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:15.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:15.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22051, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080193+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:36:08.080193+0000', 'last_peered': '2024-04-26T00:36:08.080193+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:08.080193+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:15.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:15.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21828, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080316+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:36:08.080316+0000', 'last_peered': '2024-04-26T00:36:08.080316+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:36:08.080316+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:15.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:36:15.035 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 24263, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:12.307313+0000', 'last_change': '2024-04-26T00:36:06.659921+0000', 'last_active': '2024-04-26T00:36:12.307313+0000', 'last_peered': '2024-04-26T00:36:12.307313+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:36:12.307313+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2313, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1344, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2313}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:15.035 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:15.439 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:15.439 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12203/6 objects degraded (203383.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":721,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1572204544,"bytes_avail":288338087936,"bytes_total":289910292480,"degraded_objects":12203,"degraded_total":6,"degraded_ratio":2033.833333333333,"recovering_objects_per_sec":257,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2061,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:15.454 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:15.455 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:16.563 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:16.563 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:16.575 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":406,"stamp":"2024-04-26T00:36:14.584246+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31008,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1535356,"kb_used_data":165908,"kb_used_omap":12,"kb_used_meta":1369395,"kb_avail":281580164,"statfs":{"total":289910292480,"available":288338087936,"internally_reserved":0,"allocated":169889792,"data_stored":168992534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1402260588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2061,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001264"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078754+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078754+0000","last_peered":"2024-04-26T00:36:08.078754+0000","last_clean":"2024-04-26T00:36:08.078754+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078754+0000","last_undegraded":"2024-04-26T00:36:08.078754+0000","last_fullsized":"2024-04-26T00:36:08.078754+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079408+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079408+0000","last_peered":"2024-04-26T00:36:08.079408+0000","last_clean":"2024-04-26T00:36:08.079408+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079408+0000","last_undegraded":"2024-04-26T00:36:08.079408+0000","last_fullsized":"2024-04-26T00:36:08.079408+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":24263,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:12.307313+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:12.307313+0000","last_peered":"2024-04-26T00:36:12.307313+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:12.307313+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1344,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12203,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31008,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":168587264,"data_stored":168554747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509708,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862132,"statfs":{"total":96636764160,"available":96114823168,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280655,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084427,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476980,"kb_used_data":107940,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93894860,"statfs":{"total":96636764160,"available":96148336640,"internally_reserved":0,"allocated":110530560,"data_stored":110238700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":110096384,"data_stored":110092771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:19.577 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:19.890 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:19.891 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:19.902 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-04-26T00:36:18.584853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520468,"kb_used_data":163508,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281595052,"statfs":{"total":289910292480,"available":288353333248,"internally_reserved":0,"allocated":167432192,"data_stored":166534934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1389481068},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1765,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.507368"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078754+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078754+0000","last_peered":"2024-04-26T00:36:08.078754+0000","last_clean":"2024-04-26T00:36:08.078754+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078754+0000","last_undegraded":"2024-04-26T00:36:08.078754+0000","last_fullsized":"2024-04-26T00:36:08.078754+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079408+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079408+0000","last_peered":"2024-04-26T00:36:08.079408+0000","last_clean":"2024-04-26T00:36:08.079408+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079408+0000","last_undegraded":"2024-04-26T00:36:08.079408+0000","last_fullsized":"2024-04-26T00:36:08.079408+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":25861,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:17.309594+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:17.309594+0000","last_peered":"2024-04-26T00:36:17.309594+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:17.309594+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1514,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166129664,"data_stored":166097147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497228,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874612,"statfs":{"total":96636764160,"available":96127602688,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":508686031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280656,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084428,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474572,"kb_used_data":105540,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93897268,"statfs":{"total":96636764160,"available":96150802432,"internally_reserved":0,"allocated":108072960,"data_stored":107781100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107638784,"data_stored":107635171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:19.904 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:20.218 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:20.219 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:20.231 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":408,"stamp":"2024-04-26T00:36:18.584853+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1520468,"kb_used_data":163508,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281595052,"statfs":{"total":289910292480,"available":288353333248,"internally_reserved":0,"allocated":167432192,"data_stored":166534934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1389481068},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1765,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.507368"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078754+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078754+0000","last_peered":"2024-04-26T00:36:08.078754+0000","last_clean":"2024-04-26T00:36:08.078754+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078754+0000","last_undegraded":"2024-04-26T00:36:08.078754+0000","last_fullsized":"2024-04-26T00:36:08.078754+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079408+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079408+0000","last_peered":"2024-04-26T00:36:08.079408+0000","last_clean":"2024-04-26T00:36:08.079408+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079408+0000","last_undegraded":"2024-04-26T00:36:08.079408+0000","last_fullsized":"2024-04-26T00:36:08.079408+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":25861,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:17.309594+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:17.309594+0000","last_peered":"2024-04-26T00:36:17.309594+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:17.309594+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1514,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":166129664,"data_stored":166097147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497228,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874612,"statfs":{"total":96636764160,"available":96127602688,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":508686031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280656,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084428,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474572,"kb_used_data":105540,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93897268,"statfs":{"total":96636764160,"available":96150802432,"internally_reserved":0,"allocated":108072960,"data_stored":107781100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107638784,"data_stored":107635171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:20.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:20.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22079, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080187+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:08.080187+0000', 'last_peered': '2024-04-26T00:36:08.080187+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:08.080187+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:20.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:20.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22051, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080193+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:36:08.080193+0000', 'last_peered': '2024-04-26T00:36:08.080193+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:08.080193+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:20.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:20.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21828, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080316+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:36:08.080316+0000', 'last_peered': '2024-04-26T00:36:08.080316+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:36:08.080316+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:20.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:36:20.233 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 25861, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:17.309594+0000', 'last_change': '2024-04-26T00:36:06.659921+0000', 'last_active': '2024-04-26T00:36:17.309594+0000', 'last_peered': '2024-04-26T00:36:17.309594+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:36:17.309594+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1514, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2143, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1514}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:20.234 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:20.635 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:20.636 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11404/6 objects degraded (190066.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":726,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":52,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1556959232,"bytes_avail":288353333248,"bytes_total":289910292480,"degraded_objects":11404,"degraded_total":6,"degraded_ratio":1900.666666666667,"recovering_objects_per_sec":167,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1765,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2024-04-26T00:35:18.574893+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:20.649 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:20.650 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:21.764 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:21.954 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:21.955 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-04-26T00:36:20.585175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1518068,"kb_used_data":161108,"kb_used_omap":12,"kb_used_meta":1356915,"kb_avail":281597452,"statfs":{"total":289910292480,"available":288355790848,"internally_reserved":0,"allocated":164974592,"data_stored":164077334,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1389481068},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1765,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1765,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001880"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21795,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078754+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078754+0000","last_peered":"2024-04-26T00:36:08.078754+0000","last_clean":"2024-04-26T00:36:08.078754+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078754+0000","last_undegraded":"2024-04-26T00:36:08.078754+0000","last_fullsized":"2024-04-26T00:36:08.078754+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21167,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079408+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079408+0000","last_peered":"2024-04-26T00:36:08.079408+0000","last_clean":"2024-04-26T00:36:08.079408+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079408+0000","last_undegraded":"2024-04-26T00:36:08.079408+0000","last_fullsized":"2024-04-26T00:36:08.079408+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":25861,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:17.309594+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:17.309594+0000","last_peered":"2024-04-26T00:36:17.309594+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:17.309594+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1514,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2143,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1514}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163672064,"data_stored":163639547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497228,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874612,"statfs":{"total":96636764160,"available":96127602688,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":508686031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280656,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084429,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472172,"kb_used_data":103140,"kb_used_omap":4,"kb_used_meta":369019,"kb_avail":93899668,"statfs":{"total":96636764160,"available":96153260032,"internally_reserved":0,"allocated":105615360,"data_stored":105323500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":377876176},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":105181184,"data_stored":105177571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:24.958 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:26.186 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:26.187 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:26.201 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-04-26T00:36:26.138066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32791,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507348,"kb_used_data":158708,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281608172,"statfs":{"total":289910292480,"available":288366768128,"internally_reserved":0,"allocated":162516992,"data_stored":161619734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1783,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.553514"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21796,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078797+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078797+0000","last_peered":"2024-04-26T00:36:08.078797+0000","last_clean":"2024-04-26T00:36:08.078797+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078797+0000","last_undegraded":"2024-04-26T00:36:08.078797+0000","last_fullsized":"2024-04-26T00:36:08.078797+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079446+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079446+0000","last_peered":"2024-04-26T00:36:08.079446+0000","last_clean":"2024-04-26T00:36:08.079446+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079446+0000","last_undegraded":"2024-04-26T00:36:08.079446+0000","last_fullsized":"2024-04-26T00:36:08.079446+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":27830,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:22.310443+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:22.310443+0000","last_peered":"2024-04-26T00:36:22.310443+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:22.310443+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32791,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":161214464,"data_stored":161181947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505548,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866292,"statfs":{"total":96636764160,"available":96119083008,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280657,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084430,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453132,"kb_used_data":100740,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93918708,"statfs":{"total":96636764160,"available":96172756992,"internally_reserved":0,"allocated":103157760,"data_stored":102865900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102723584,"data_stored":102719971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:26.202 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:26.550 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:26.551 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:26.563 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":413,"stamp":"2024-04-26T00:36:26.138066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32791,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507348,"kb_used_data":158708,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281608172,"statfs":{"total":289910292480,"available":288366768128,"internally_reserved":0,"allocated":162516992,"data_stored":161619734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1783,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1783,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.553514"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21796,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078797+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078797+0000","last_peered":"2024-04-26T00:36:08.078797+0000","last_clean":"2024-04-26T00:36:08.078797+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078797+0000","last_undegraded":"2024-04-26T00:36:08.078797+0000","last_fullsized":"2024-04-26T00:36:08.078797+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079446+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079446+0000","last_peered":"2024-04-26T00:36:08.079446+0000","last_clean":"2024-04-26T00:36:08.079446+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079446+0000","last_undegraded":"2024-04-26T00:36:08.079446+0000","last_fullsized":"2024-04-26T00:36:08.079446+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":27830,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:22.310443+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:22.310443+0000","last_peered":"2024-04-26T00:36:22.310443+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:22.310443+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32791,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":161214464,"data_stored":161181947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505548,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866292,"statfs":{"total":96636764160,"available":96119083008,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280657,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084430,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453132,"kb_used_data":100740,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93918708,"statfs":{"total":96636764160,"available":96172756992,"internally_reserved":0,"allocated":103157760,"data_stored":102865900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102723584,"data_stored":102719971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:26.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:26.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22079, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080187+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:08.080187+0000', 'last_peered': '2024-04-26T00:36:08.080187+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:08.080187+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:26.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:26.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22051, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080193+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:36:08.080193+0000', 'last_peered': '2024-04-26T00:36:08.080193+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:08.080193+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:26.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:26.564 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 21828, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:08.080316+0000', 'last_change': '2024-04-26T00:26:01.847170+0000', 'last_active': '2024-04-26T00:36:08.080316+0000', 'last_peered': '2024-04-26T00:36:08.080316+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:36:08.080316+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3306, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 251, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3303}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:26.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-04-26T00:36:26.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "44'21420", 'reported_seq': 27830, 'reported_epoch': 52, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:22.310443+0000', 'last_change': '2024-04-26T00:36:06.659921+0000', 'last_active': '2024-04-26T00:36:22.310443+0000', 'last_peered': '2024-04-26T00:36:22.310443+0000', 'last_clean': '2024-04-26T00:25:59.337353+0000', 'last_became_active': '2024-04-26T00:26:01.834001+0000', 'last_became_peered': '2024-04-26T00:26:01.834001+0000', 'last_unstale': '2024-04-26T00:36:22.310443+0000', 'last_undegraded': '2024-04-26T00:26:01.434440+0000', 'last_fullsized': '2024-04-26T00:26:01.359337+0000', 'mapping_epoch': 35, 'log_start': "38'11400", 'ondisk_log_start': "38'11400", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3325", 'last_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:53.106632+0000', 'objects_scrubbed': 3300, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1077, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 530, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18116, 'num_write_kb': 29632, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3127, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:26.565 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:26.994 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:26.995 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10420/6 objects degraded (173666.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":732,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1563021312,"bytes_avail":288347271168,"bytes_total":289910292480,"degraded_objects":10420,"degraded_total":6,"degraded_ratio":1736.666666666667,"recovering_objects_per_sec":148,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1783,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:27.010 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:27.010 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:27.409 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:27.409 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:27.421 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-04-26T00:36:26.586086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32791,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1507348,"kb_used_data":158708,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281608172,"statfs":{"total":289910292480,"available":288366768128,"internally_reserved":0,"allocated":162516992,"data_stored":161619734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-984,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":984,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001232"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28625,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081983+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:08.081983+0000","last_peered":"2024-04-26T00:36:08.081983+0000","last_clean":"2024-04-26T00:36:08.081983+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:08.081983+0000","last_undegraded":"2024-04-26T00:36:08.081983+0000","last_fullsized":"2024-04-26T00:36:08.081983+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21796,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078797+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:08.078797+0000","last_peered":"2024-04-26T00:36:08.078797+0000","last_clean":"2024-04-26T00:36:08.078797+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:08.078797+0000","last_undegraded":"2024-04-26T00:36:08.078797+0000","last_fullsized":"2024-04-26T00:36:08.078797+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28578,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081773+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:08.081773+0000","last_peered":"2024-04-26T00:36:08.081773+0000","last_clean":"2024-04-26T00:36:08.081773+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:08.081773+0000","last_undegraded":"2024-04-26T00:36:08.081773+0000","last_fullsized":"2024-04-26T00:36:08.081773+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21168,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079446+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:08.079446+0000","last_peered":"2024-04-26T00:36:08.079446+0000","last_clean":"2024-04-26T00:36:08.079446+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:08.079446+0000","last_undegraded":"2024-04-26T00:36:08.079446+0000","last_fullsized":"2024-04-26T00:36:08.079446+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22013,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080314+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:08.080314+0000","last_peered":"2024-04-26T00:36:08.080314+0000","last_clean":"2024-04-26T00:36:08.080314+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:08.080314+0000","last_undegraded":"2024-04-26T00:36:08.080314+0000","last_fullsized":"2024-04-26T00:36:08.080314+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22079,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080187+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:08.080187+0000","last_peered":"2024-04-26T00:36:08.080187+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:08.080187+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28326,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081582+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:08.081582+0000","last_peered":"2024-04-26T00:36:08.081582+0000","last_clean":"2024-04-26T00:36:08.081582+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:08.081582+0000","last_undegraded":"2024-04-26T00:36:08.081582+0000","last_fullsized":"2024-04-26T00:36:08.081582+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21633,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080013+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:08.080013+0000","last_peered":"2024-04-26T00:36:08.080013+0000","last_clean":"2024-04-26T00:36:08.080013+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:08.080013+0000","last_undegraded":"2024-04-26T00:36:08.080013+0000","last_fullsized":"2024-04-26T00:36:08.080013+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081443+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:08.081443+0000","last_peered":"2024-04-26T00:36:08.081443+0000","last_clean":"2024-04-26T00:36:08.081443+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:08.081443+0000","last_undegraded":"2024-04-26T00:36:08.081443+0000","last_fullsized":"2024-04-26T00:36:08.081443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21649,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081802+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:08.081802+0000","last_peered":"2024-04-26T00:36:08.081802+0000","last_clean":"2024-04-26T00:36:08.081802+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:08.081802+0000","last_undegraded":"2024-04-26T00:36:08.081802+0000","last_fullsized":"2024-04-26T00:36:08.081802+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:12.163473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079147+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:08.079147+0000","last_peered":"2024-04-26T00:36:08.079147+0000","last_clean":"2024-04-26T00:36:08.079147+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:08.079147+0000","last_undegraded":"2024-04-26T00:36:08.079147+0000","last_fullsized":"2024-04-26T00:36:08.079147+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28827,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.134769+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:08.134769+0000","last_peered":"2024-04-26T00:36:08.134769+0000","last_clean":"2024-04-26T00:36:08.134769+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:08.134769+0000","last_undegraded":"2024-04-26T00:36:08.134769+0000","last_fullsized":"2024-04-26T00:36:08.134769+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.080078+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:08.080078+0000","last_peered":"2024-04-26T00:36:08.080078+0000","last_clean":"2024-04-26T00:36:08.080078+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:08.080078+0000","last_undegraded":"2024-04-26T00:36:08.080078+0000","last_fullsized":"2024-04-26T00:36:08.080078+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28906,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.079266+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:08.079266+0000","last_peered":"2024-04-26T00:36:08.079266+0000","last_clean":"2024-04-26T00:36:08.079266+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:08.079266+0000","last_undegraded":"2024-04-26T00:36:08.079266+0000","last_fullsized":"2024-04-26T00:36:08.079266+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081724+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:08.081724+0000","last_peered":"2024-04-26T00:36:08.081724+0000","last_clean":"2024-04-26T00:36:08.081724+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:08.081724+0000","last_undegraded":"2024-04-26T00:36:08.081724+0000","last_fullsized":"2024-04-26T00:36:08.081724+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28340,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078832+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:08.078832+0000","last_peered":"2024-04-26T00:36:08.078832+0000","last_clean":"2024-04-26T00:36:08.078832+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:08.078832+0000","last_undegraded":"2024-04-26T00:36:08.078832+0000","last_fullsized":"2024-04-26T00:36:08.078832+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22051,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080193+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:08.080193+0000","last_peered":"2024-04-26T00:36:08.080193+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:08.080193+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081914+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:08.081914+0000","last_peered":"2024-04-26T00:36:08.081914+0000","last_clean":"2024-04-26T00:36:08.081914+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:08.081914+0000","last_undegraded":"2024-04-26T00:36:08.081914+0000","last_fullsized":"2024-04-26T00:36:08.081914+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21828,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:08.080316+0000","last_change":"2024-04-26T00:26:01.847170+0000","last_active":"2024-04-26T00:36:08.080316+0000","last_peered":"2024-04-26T00:36:08.080316+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:08.080316+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":251,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":71,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.078706+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:08.078706+0000","last_peered":"2024-04-26T00:36:08.078706+0000","last_clean":"2024-04-26T00:36:08.078706+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:08.078706+0000","last_undegraded":"2024-04-26T00:36:08.078706+0000","last_fullsized":"2024-04-26T00:36:08.078706+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":27830,"reported_epoch":52,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:22.310443+0000","last_change":"2024-04-26T00:36:06.659921+0000","last_active":"2024-04-26T00:36:22.310443+0000","last_peered":"2024-04-26T00:36:22.310443+0000","last_clean":"2024-04-26T00:25:59.337353+0000","last_became_active":"2024-04-26T00:26:01.834001+0000","last_became_peered":"2024-04-26T00:26:01.834001+0000","last_unstale":"2024-04-26T00:36:22.310443+0000","last_undegraded":"2024-04-26T00:26:01.434440+0000","last_fullsized":"2024-04-26T00:26:01.359337+0000","mapping_epoch":35,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3127,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.082007+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:08.082007+0000","last_peered":"2024-04-26T00:36:08.082007+0000","last_clean":"2024-04-26T00:36:08.082007+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:08.082007+0000","last_undegraded":"2024-04-26T00:36:08.082007+0000","last_fullsized":"2024-04-26T00:36:08.082007+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081886+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:08.081886+0000","last_peered":"2024-04-26T00:36:08.081886+0000","last_clean":"2024-04-26T00:36:08.081886+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:08.081886+0000","last_undegraded":"2024-04-26T00:36:08.081886+0000","last_fullsized":"2024-04-26T00:36:08.081886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28755,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-04-26T00:36:08.081821+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:08.081821+0000","last_peered":"2024-04-26T00:36:08.081821+0000","last_clean":"2024-04-26T00:36:08.081821+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:08.081821+0000","last_undegraded":"2024-04-26T00:36:08.081821+0000","last_fullsized":"2024-04-26T00:36:08.081821+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10420,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32791,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":161214464,"data_stored":161181947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505548,"kb_used_data":452,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866292,"statfs":{"total":96636764160,"available":96119083008,"internally_reserved":0,"allocated":462848,"data_stored":149310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280657,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548668,"kb_used_data":57516,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823172,"statfs":{"total":96636764160,"available":96074928128,"internally_reserved":0,"allocated":58896384,"data_stored":58604524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084430,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":453132,"kb_used_data":100740,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93918708,"statfs":{"total":96636764160,"available":96172756992,"internally_reserved":0,"allocated":103157760,"data_stored":102865900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102723584,"data_stored":102719971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:30.423 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:30.756 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:30.757 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:30.769 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-04-26T00:36:30.586782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33756,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496188,"kb_used_data":155868,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281619332,"statfs":{"total":289910292480,"available":288378195968,"internally_reserved":0,"allocated":159608832,"data_stored":158690028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1372441708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-965,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001313"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28330,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141381+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141381+0000","last_peered":"2024-04-26T00:36:26.141381+0000","last_clean":"2024-04-26T00:36:26.141381+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141381+0000","last_undegraded":"2024-04-26T00:36:26.141381+0000","last_fullsized":"2024-04-26T00:36:26.141381+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":22703,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:27.312172+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:27.312172+0000","last_peered":"2024-04-26T00:36:27.312172+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:27.312172+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2871}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33756,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158257152,"data_stored":158235474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497232,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874608,"statfs":{"total":96636764160,"available":96127598592,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":508686031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280658,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084431,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450272,"kb_used_data":97880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93921568,"statfs":{"total":96636764160,"available":96175685632,"internally_reserved":0,"allocated":100229120,"data_stored":99926465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99778560,"data_stored":99774947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:30.770 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:31.073 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:31.074 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:31.085 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-04-26T00:36:30.586782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33756,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496188,"kb_used_data":155868,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281619332,"statfs":{"total":289910292480,"available":288378195968,"internally_reserved":0,"allocated":159608832,"data_stored":158690028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1372441708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-965,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001313"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28330,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141381+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141381+0000","last_peered":"2024-04-26T00:36:26.141381+0000","last_clean":"2024-04-26T00:36:26.141381+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141381+0000","last_undegraded":"2024-04-26T00:36:26.141381+0000","last_fullsized":"2024-04-26T00:36:26.141381+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":22703,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:27.312172+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:27.312172+0000","last_peered":"2024-04-26T00:36:27.312172+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:27.312172+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2871}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33756,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158257152,"data_stored":158235474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497232,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874608,"statfs":{"total":96636764160,"available":96127598592,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":508686031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280658,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084431,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450272,"kb_used_data":97880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93921568,"statfs":{"total":96636764160,"available":96175685632,"internally_reserved":0,"allocated":100229120,"data_stored":99926465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99778560,"data_stored":99774947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:31.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:31.086 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22083, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:26.143312+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:26.143312+0000', 'last_peered': '2024-04-26T00:36:26.143312+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:26.143312+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:31.087 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:31.087 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22055, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:26.143748+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:36:26.143748+0000', 'last_peered': '2024-04-26T00:36:26.143748+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:26.143748+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:31.087 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:31.087 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 22703, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:27.312172+0000', 'last_change': '2024-04-26T00:36:24.364966+0000', 'last_active': '2024-04-26T00:36:27.312172+0000', 'last_peered': '2024-04-26T00:36:27.312172+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:36:27.312172+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2871, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 686, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2871}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:31.087 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:31.433 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:31.433 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9455/6 objects degraded (157583.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":737,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1532096512,"bytes_avail":288378195968,"bytes_total":289910292480,"degraded_objects":9455,"degraded_total":6,"degraded_ratio":1575.833333333333,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:31.446 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:31.446 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:31.747 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:31.747 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:31.761 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-04-26T00:36:30.586782+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33756,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496188,"kb_used_data":155868,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281619332,"statfs":{"total":289910292480,"available":288378195968,"internally_reserved":0,"allocated":159608832,"data_stored":158690028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1372441708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-965,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001313"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28330,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141381+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141381+0000","last_peered":"2024-04-26T00:36:26.141381+0000","last_clean":"2024-04-26T00:36:26.141381+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141381+0000","last_undegraded":"2024-04-26T00:36:26.141381+0000","last_fullsized":"2024-04-26T00:36:26.141381+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":22703,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:27.312172+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:27.312172+0000","last_peered":"2024-04-26T00:36:27.312172+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:27.312172+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2871,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":686,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2871}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33756,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158257152,"data_stored":158235474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497232,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874608,"statfs":{"total":96636764160,"available":96127598592,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":508686031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280658,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084431,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450272,"kb_used_data":97880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93921568,"statfs":{"total":96636764160,"available":96175685632,"internally_reserved":0,"allocated":100229120,"data_stored":99926465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99778560,"data_stored":99774947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:34.764 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:35.064 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:35.065 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:35.077 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-04-26T00:36:34.587460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34315,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500348,"kb_used_data":155868,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281615172,"statfs":{"total":289910292480,"available":288373936128,"internally_reserved":0,"allocated":159608832,"data_stored":158690028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.448554"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":23821,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:32.307941+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:32.307941+0000","last_peered":"2024-04-26T00:36:32.307941+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:32.307941+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34315,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158257152,"data_stored":158235474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501392,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870448,"statfs":{"total":96636764160,"available":96123338752,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280659,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084431,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450272,"kb_used_data":97880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93921568,"statfs":{"total":96636764160,"available":96175685632,"internally_reserved":0,"allocated":100229120,"data_stored":99926465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99778560,"data_stored":99774947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:35.078 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:35.410 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:35.410 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:35.423 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-04-26T00:36:34.587460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34315,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500348,"kb_used_data":155868,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281615172,"statfs":{"total":289910292480,"available":288373936128,"internally_reserved":0,"allocated":159608832,"data_stored":158690028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.448554"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":23821,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:32.307941+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:32.307941+0000","last_peered":"2024-04-26T00:36:32.307941+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:32.307941+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34315,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158257152,"data_stored":158235474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501392,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870448,"statfs":{"total":96636764160,"available":96123338752,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280659,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084431,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450272,"kb_used_data":97880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93921568,"statfs":{"total":96636764160,"available":96175685632,"internally_reserved":0,"allocated":100229120,"data_stored":99926465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99778560,"data_stored":99774947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:35.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:35.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22083, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:26.143312+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:26.143312+0000', 'last_peered': '2024-04-26T00:36:26.143312+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:26.143312+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:35.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:35.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22055, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:26.143748+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:36:26.143748+0000', 'last_peered': '2024-04-26T00:36:26.143748+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:26.143748+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:35.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:35.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 23821, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:32.307941+0000', 'last_change': '2024-04-26T00:36:24.364966+0000', 'last_active': '2024-04-26T00:36:32.307941+0000', 'last_peered': '2024-04-26T00:36:32.307941+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:36:32.307941+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2312, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1245, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2312}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:35.425 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:35.834 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:35.834 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8896/6 objects degraded (148266.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":741,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1536356352,"bytes_avail":288373936128,"bytes_total":289910292480,"degraded_objects":8896,"degraded_total":6,"degraded_ratio":1482.666666666667,"recovering_objects_per_sec":161,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1524,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:35.848 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:35.848 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:36.144 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:36.145 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:36.159 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-04-26T00:36:34.587460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34315,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1500348,"kb_used_data":155868,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281615172,"statfs":{"total":289910292480,"available":288373936128,"internally_reserved":0,"allocated":159608832,"data_stored":158690028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1524,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.448554"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":23821,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:32.307941+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:32.307941+0000","last_peered":"2024-04-26T00:36:32.307941+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:32.307941+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2312,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2312}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34315,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":158257152,"data_stored":158235474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501392,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870448,"statfs":{"total":96636764160,"available":96123338752,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280659,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084431,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450272,"kb_used_data":97880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93921568,"statfs":{"total":96636764160,"available":96175685632,"internally_reserved":0,"allocated":100229120,"data_stored":99926465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":99778560,"data_stored":99774947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:39.163 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:39.472 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:39.472 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:39.484 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-04-26T00:36:38.588055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35096,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505668,"kb_used_data":152868,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281609852,"statfs":{"total":289910292480,"available":288368488448,"internally_reserved":0,"allocated":156536832,"data_stored":155618028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001968"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":25383,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:37.312728+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:37.312728+0000","last_peered":"2024-04-26T00:36:37.312728+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:37.312728+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1531,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35096,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155185152,"data_stored":155163474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280660,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084432,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447272,"kb_used_data":94880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93924568,"statfs":{"total":96636764160,"available":96178757632,"internally_reserved":0,"allocated":97157120,"data_stored":96854465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":96702947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:39.485 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:39.801 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:39.802 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:39.813 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-04-26T00:36:38.588055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35096,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505668,"kb_used_data":152868,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281609852,"statfs":{"total":289910292480,"available":288368488448,"internally_reserved":0,"allocated":156536832,"data_stored":155618028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001968"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":25383,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:37.312728+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:37.312728+0000","last_peered":"2024-04-26T00:36:37.312728+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:37.312728+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1531,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35096,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155185152,"data_stored":155163474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280660,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084432,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447272,"kb_used_data":94880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93924568,"statfs":{"total":96636764160,"available":96178757632,"internally_reserved":0,"allocated":97157120,"data_stored":96854465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":96702947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:39.814 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:39.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22083, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:26.143312+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:26.143312+0000', 'last_peered': '2024-04-26T00:36:26.143312+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:26.143312+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:39.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:39.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22055, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:26.143748+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:36:26.143748+0000', 'last_peered': '2024-04-26T00:36:26.143748+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:26.143748+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:39.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:39.815 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 25383, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:37.312728+0000', 'last_change': '2024-04-26T00:36:24.364966+0000', 'last_active': '2024-04-26T00:36:37.312728+0000', 'last_peered': '2024-04-26T00:36:37.312728+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:36:37.312728+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1531, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2026, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1531}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:39.815 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:40.206 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:40.207 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8115/6 objects degraded (135250.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":746,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1541804032,"bytes_avail":288368488448,"bytes_total":289910292480,"degraded_objects":8115,"degraded_total":6,"degraded_ratio":1352.5,"recovering_objects_per_sec":192,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2305,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:40.219 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:40.220 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:40.545 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:40.546 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:40.557 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-04-26T00:36:38.588055+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35096,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1505668,"kb_used_data":152868,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281609852,"statfs":{"total":289910292480,"available":288368488448,"internally_reserved":0,"allocated":156536832,"data_stored":155618028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2305,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2305,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001968"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21637,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143138+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143138+0000","last_peered":"2024-04-26T00:36:26.143138+0000","last_clean":"2024-04-26T00:36:26.143138+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143138+0000","last_undegraded":"2024-04-26T00:36:26.143138+0000","last_fullsized":"2024-04-26T00:36:26.143138+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":25383,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:37.312728+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:37.312728+0000","last_peered":"2024-04-26T00:36:37.312728+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:37.312728+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1531,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1531}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8115,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35096,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155185152,"data_stored":155163474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280660,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084432,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":447272,"kb_used_data":94880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93924568,"statfs":{"total":96636764160,"available":96178757632,"internally_reserved":0,"allocated":97157120,"data_stored":96854465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":96706560,"data_stored":96702947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:43.559 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:43.863 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:43.864 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:43.876 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-26T00:36:42.588655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36039,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1502668,"kb_used_data":149868,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281612852,"statfs":{"total":289910292480,"available":288371560448,"internally_reserved":0,"allocated":153464832,"data_stored":152546028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001872"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21638,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143193+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143193+0000","last_peered":"2024-04-26T00:36:26.143193+0000","last_clean":"2024-04-26T00:36:26.143193+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143193+0000","last_undegraded":"2024-04-26T00:36:26.143193+0000","last_fullsized":"2024-04-26T00:36:26.143193+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":27269,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:42.314786+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:42.314786+0000","last_peered":"2024-04-26T00:36:42.314786+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:42.314786+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36039,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152113152,"data_stored":152091474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280661,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084433,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444272,"kb_used_data":91880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93927568,"statfs":{"total":96636764160,"available":96181829632,"internally_reserved":0,"allocated":94085120,"data_stored":93782465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":93634560,"data_stored":93630947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:43.877 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:44.171 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:44.172 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:44.184 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":422,"stamp":"2024-04-26T00:36:42.588655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36039,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1502668,"kb_used_data":149868,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281612852,"statfs":{"total":289910292480,"available":288371560448,"internally_reserved":0,"allocated":153464832,"data_stored":152546028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001872"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21638,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143193+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143193+0000","last_peered":"2024-04-26T00:36:26.143193+0000","last_clean":"2024-04-26T00:36:26.143193+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143193+0000","last_undegraded":"2024-04-26T00:36:26.143193+0000","last_fullsized":"2024-04-26T00:36:26.143193+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":27269,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:42.314786+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:42.314786+0000","last_peered":"2024-04-26T00:36:42.314786+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:42.314786+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":91,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141283+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141283+0000","last_peered":"2024-04-26T00:36:26.141283+0000","last_clean":"2024-04-26T00:36:26.141283+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141283+0000","last_undegraded":"2024-04-26T00:36:26.141283+0000","last_fullsized":"2024-04-26T00:36:26.141283+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36039,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152113152,"data_stored":152091474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509712,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862128,"statfs":{"total":96636764160,"available":96114819072,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280661,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084433,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444272,"kb_used_data":91880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93927568,"statfs":{"total":96636764160,"available":96181829632,"internally_reserved":0,"allocated":94085120,"data_stored":93782465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":93634560,"data_stored":93630947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:44.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:44.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22083, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:26.143312+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:26.143312+0000', 'last_peered': '2024-04-26T00:36:26.143312+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:26.143312+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:44.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:44.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22055, 'reported_epoch': 54, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:26.143748+0000', 'last_change': '2024-04-26T00:26:01.849875+0000', 'last_active': '2024-04-26T00:36:26.143748+0000', 'last_peered': '2024-04-26T00:36:26.143748+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:26.143748+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3336, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:44.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-04-26T00:36:44.186 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "44'21170", 'reported_seq': 27269, 'reported_epoch': 54, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:42.314786+0000', 'last_change': '2024-04-26T00:36:24.364966+0000', 'last_active': '2024-04-26T00:36:42.314786+0000', 'last_peered': '2024-04-26T00:36:42.314786+0000', 'last_clean': '2024-04-26T00:25:59.337047+0000', 'last_became_active': '2024-04-26T00:26:01.834914+0000', 'last_became_peered': '2024-04-26T00:26:01.834914+0000', 'last_unstale': '2024-04-26T00:36:42.314786+0000', 'last_undegraded': '2024-04-26T00:26:01.432829+0000', 'last_fullsized': '2024-04-26T00:26:01.360103+0000', 'mapping_epoch': 35, 'log_start': "38'11100", 'ondisk_log_start': "38'11100", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3145", 'last_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:50.799180+0000', 'objects_scrubbed': 3110, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 840, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 588, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18052, 'num_write_kb': 29884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2969, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -588}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:44.186 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:44.552 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:44.552 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7172/6 objects degraded (119533.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":750,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":54,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1538732032,"bytes_avail":288371560448,"bytes_total":289910292480,"degraded_objects":7172,"degraded_total":6,"degraded_ratio":1195.3333333333333,"recovering_objects_per_sec":190,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2283,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:44.565 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:44.565 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:44.908 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:44.908 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:44.923 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-04-26T00:36:44.588977+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36039,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494348,"kb_used_data":149868,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281621172,"statfs":{"total":289910292480,"available":288380080128,"internally_reserved":0,"allocated":153464832,"data_stored":152546028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1724,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001899"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21638,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143193+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143193+0000","last_peered":"2024-04-26T00:36:26.143193+0000","last_clean":"2024-04-26T00:36:26.143193+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143193+0000","last_undegraded":"2024-04-26T00:36:26.143193+0000","last_fullsized":"2024-04-26T00:36:26.143193+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":27269,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:42.314786+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:42.314786+0000","last_peered":"2024-04-26T00:36:42.314786+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:42.314786+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141323+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141323+0000","last_peered":"2024-04-26T00:36:26.141323+0000","last_clean":"2024-04-26T00:36:26.141323+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141323+0000","last_undegraded":"2024-04-26T00:36:26.141323+0000","last_fullsized":"2024-04-26T00:36:26.141323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36039,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":152113152,"data_stored":152091474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501392,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870448,"statfs":{"total":96636764160,"available":96123338752,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280661,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084433,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":444272,"kb_used_data":91880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93927568,"statfs":{"total":96636764160,"available":96181829632,"internally_reserved":0,"allocated":94085120,"data_stored":93782465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":93634560,"data_stored":93630947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:47.925 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:48.277 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:48.277 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:48.291 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":426,"stamp":"2024-04-26T00:36:46.589293+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36039,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1491348,"kb_used_data":146868,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281624172,"statfs":{"total":289910292480,"available":288383152128,"internally_reserved":0,"allocated":150392832,"data_stored":149474028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-943,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001238"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28629,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143747+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:26.143747+0000","last_peered":"2024-04-26T00:36:26.143747+0000","last_clean":"2024-04-26T00:36:26.143747+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:26.143747+0000","last_undegraded":"2024-04-26T00:36:26.143747+0000","last_fullsized":"2024-04-26T00:36:26.143747+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28583,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141368+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:26.141368+0000","last_peered":"2024-04-26T00:36:26.141368+0000","last_clean":"2024-04-26T00:36:26.141368+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:26.141368+0000","last_undegraded":"2024-04-26T00:36:26.141368+0000","last_fullsized":"2024-04-26T00:36:26.141368+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22017,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143284+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:26.143284+0000","last_peered":"2024-04-26T00:36:26.143284+0000","last_clean":"2024-04-26T00:36:26.143284+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:26.143284+0000","last_undegraded":"2024-04-26T00:36:26.143284+0000","last_fullsized":"2024-04-26T00:36:26.143284+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143312+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:26.143312+0000","last_peered":"2024-04-26T00:36:26.143312+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:26.143312+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28331,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141424+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:26.141424+0000","last_peered":"2024-04-26T00:36:26.141424+0000","last_clean":"2024-04-26T00:36:26.141424+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:26.141424+0000","last_undegraded":"2024-04-26T00:36:26.141424+0000","last_fullsized":"2024-04-26T00:36:26.141424+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21638,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143193+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:26.143193+0000","last_peered":"2024-04-26T00:36:26.143193+0000","last_clean":"2024-04-26T00:36:26.143193+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:26.143193+0000","last_undegraded":"2024-04-26T00:36:26.143193+0000","last_fullsized":"2024-04-26T00:36:26.143193+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141202+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:26.141202+0000","last_peered":"2024-04-26T00:36:26.141202+0000","last_clean":"2024-04-26T00:36:26.141202+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:26.141202+0000","last_undegraded":"2024-04-26T00:36:26.141202+0000","last_fullsized":"2024-04-26T00:36:26.141202+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21653,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141498+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:26.141498+0000","last_peered":"2024-04-26T00:36:26.141498+0000","last_clean":"2024-04-26T00:36:26.141498+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:26.141498+0000","last_undegraded":"2024-04-26T00:36:26.141498+0000","last_fullsized":"2024-04-26T00:36:26.141498+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:29.193969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28831,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143582+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:26.143582+0000","last_peered":"2024-04-26T00:36:26.143582+0000","last_clean":"2024-04-26T00:36:26.143582+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:26.143582+0000","last_undegraded":"2024-04-26T00:36:26.143582+0000","last_fullsized":"2024-04-26T00:36:26.143582+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":102,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.143218+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:26.143218+0000","last_peered":"2024-04-26T00:36:26.143218+0000","last_clean":"2024-04-26T00:36:26.143218+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:26.143218+0000","last_undegraded":"2024-04-26T00:36:26.143218+0000","last_fullsized":"2024-04-26T00:36:26.143218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":96,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141525+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:26.141525+0000","last_peered":"2024-04-26T00:36:26.141525+0000","last_clean":"2024-04-26T00:36:26.141525+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:26.141525+0000","last_undegraded":"2024-04-26T00:36:26.141525+0000","last_fullsized":"2024-04-26T00:36:26.141525+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22055,"reported_epoch":54,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:26.143748+0000","last_change":"2024-04-26T00:26:01.849875+0000","last_active":"2024-04-26T00:36:26.143748+0000","last_peered":"2024-04-26T00:36:26.143748+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:26.143748+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3336,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141628+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:26.141628+0000","last_peered":"2024-04-26T00:36:26.141628+0000","last_clean":"2024-04-26T00:36:26.141628+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:26.141628+0000","last_undegraded":"2024-04-26T00:36:26.141628+0000","last_fullsized":"2024-04-26T00:36:26.141628+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":27269,"reported_epoch":54,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:42.314786+0000","last_change":"2024-04-26T00:36:24.364966+0000","last_active":"2024-04-26T00:36:42.314786+0000","last_peered":"2024-04-26T00:36:42.314786+0000","last_clean":"2024-04-26T00:25:59.337047+0000","last_became_active":"2024-04-26T00:26:01.834914+0000","last_became_peered":"2024-04-26T00:26:01.834914+0000","last_unstale":"2024-04-26T00:36:42.314786+0000","last_undegraded":"2024-04-26T00:26:01.432829+0000","last_fullsized":"2024-04-26T00:26:01.360103+0000","mapping_epoch":35,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":588,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2969,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22151,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.244060+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:26.244060+0000","last_peered":"2024-04-26T00:36:26.244060+0000","last_clean":"2024-04-26T00:36:26.244060+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:26.244060+0000","last_undegraded":"2024-04-26T00:36:26.244060+0000","last_fullsized":"2024-04-26T00:36:26.244060+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":92,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141323+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:26.141323+0000","last_peered":"2024-04-26T00:36:26.141323+0000","last_clean":"2024-04-26T00:36:26.141323+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:26.141323+0000","last_undegraded":"2024-04-26T00:36:26.141323+0000","last_fullsized":"2024-04-26T00:36:26.141323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":95,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141686+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:26.141686+0000","last_peered":"2024-04-26T00:36:26.141686+0000","last_clean":"2024-04-26T00:36:26.141686+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:26.141686+0000","last_undegraded":"2024-04-26T00:36:26.141686+0000","last_fullsized":"2024-04-26T00:36:26.141686+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28759,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.141591+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:26.141591+0000","last_peered":"2024-04-26T00:36:26.141591+0000","last_clean":"2024-04-26T00:36:26.141591+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:26.141591+0000","last_undegraded":"2024-04-26T00:36:26.141591+0000","last_fullsized":"2024-04-26T00:36:26.141591+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36039,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":149041152,"data_stored":149019474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501392,"kb_used_data":456,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870448,"statfs":{"total":96636764160,"available":96123338752,"internally_reserved":0,"allocated":466944,"data_stored":153450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280661,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548684,"kb_used_data":57532,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823156,"statfs":{"total":96636764160,"available":96074911744,"internally_reserved":0,"allocated":58912768,"data_stored":58610113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084434,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441272,"kb_used_data":88880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93930568,"statfs":{"total":96636764160,"available":96184901632,"internally_reserved":0,"allocated":91013120,"data_stored":90710465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90562560,"data_stored":90558947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:48.292 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:48.632 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:48.632 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:48.646 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-04-26T00:36:48.589618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37049,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1499700,"kb_used_data":146896,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281615820,"statfs":{"total":289910292480,"available":288374599680,"internally_reserved":0,"allocated":150421504,"data_stored":149484623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001250"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208867+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208867+0000","last_peered":"2024-04-26T00:36:46.208867+0000","last_clean":"2024-04-26T00:36:46.208867+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208867+0000","last_undegraded":"2024-04-26T00:36:46.208867+0000","last_fullsized":"2024-04-26T00:36:46.208867+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22904,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:47.314931+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:36:47.314931+0000","last_peered":"2024-04-26T00:36:47.314931+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:47.314931+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207709+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207709+0000","last_peered":"2024-04-26T00:36:46.207709+0000","last_clean":"2024-04-26T00:36:46.207709+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207709+0000","last_undegraded":"2024-04-26T00:36:46.207709+0000","last_fullsized":"2024-04-26T00:36:46.207709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37049,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":149037056,"data_stored":149018991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280662,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084434,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441272,"kb_used_data":88880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93930568,"statfs":{"total":96636764160,"available":96184901632,"internally_reserved":0,"allocated":91013120,"data_stored":90710465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90562560,"data_stored":90558947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:48.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:48.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22087, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:46.209587+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:46.209587+0000', 'last_peered': '2024-04-26T00:36:46.209587+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:46.209587+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:48.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:48.648 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 22904, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:47.314931+0000', 'last_change': '2024-04-26T00:36:44.574130+0000', 'last_active': '2024-04-26T00:36:47.314931+0000', 'last_peered': '2024-04-26T00:36:47.314931+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:47.314931+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2914, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 722, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2914}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:48.648 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:49.164 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:49.164 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7172/6 objects degraded (119533.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":755,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1527140352,"bytes_avail":288383152128,"bytes_total":289910292480,"degraded_objects":7172,"degraded_total":6,"degraded_ratio":1195.3333333333333,"recovering_objects_per_sec":117,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:49.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:49.179 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:49.527 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:49.527 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:49.540 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":427,"stamp":"2024-04-26T00:36:48.589618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37049,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1499700,"kb_used_data":146896,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281615820,"statfs":{"total":289910292480,"available":288374599680,"internally_reserved":0,"allocated":150421504,"data_stored":149484623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001250"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21800,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139649+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:26.139649+0000","last_peered":"2024-04-26T00:36:26.139649+0000","last_clean":"2024-04-26T00:36:26.139649+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:26.139649+0000","last_undegraded":"2024-04-26T00:36:26.139649+0000","last_fullsized":"2024-04-26T00:36:26.139649+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21172,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139294+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:26.139294+0000","last_peered":"2024-04-26T00:36:26.139294+0000","last_clean":"2024-04-26T00:36:26.139294+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:26.139294+0000","last_undegraded":"2024-04-26T00:36:26.139294+0000","last_fullsized":"2024-04-26T00:36:26.139294+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208867+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208867+0000","last_peered":"2024-04-26T00:36:46.208867+0000","last_clean":"2024-04-26T00:36:46.208867+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208867+0000","last_undegraded":"2024-04-26T00:36:46.208867+0000","last_fullsized":"2024-04-26T00:36:46.208867+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138998+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:26.138998+0000","last_peered":"2024-04-26T00:36:26.138998+0000","last_clean":"2024-04-26T00:36:26.138998+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:26.138998+0000","last_undegraded":"2024-04-26T00:36:26.138998+0000","last_fullsized":"2024-04-26T00:36:26.138998+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28910,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139090+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:26.139090+0000","last_peered":"2024-04-26T00:36:26.139090+0000","last_clean":"2024-04-26T00:36:26.139090+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:26.139090+0000","last_undegraded":"2024-04-26T00:36:26.139090+0000","last_fullsized":"2024-04-26T00:36:26.139090+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28344,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.138968+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:26.138968+0000","last_peered":"2024-04-26T00:36:26.138968+0000","last_clean":"2024-04-26T00:36:26.138968+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:26.138968+0000","last_undegraded":"2024-04-26T00:36:26.138968+0000","last_fullsized":"2024-04-26T00:36:26.138968+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22904,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:47.314931+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:36:47.314931+0000","last_peered":"2024-04-26T00:36:47.314931+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:47.314931+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2914,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-26T00:36:26.139560+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:26.139560+0000","last_peered":"2024-04-26T00:36:26.139560+0000","last_clean":"2024-04-26T00:36:26.139560+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:26.139560+0000","last_undegraded":"2024-04-26T00:36:26.139560+0000","last_fullsized":"2024-04-26T00:36:26.139560+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207709+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207709+0000","last_peered":"2024-04-26T00:36:46.207709+0000","last_clean":"2024-04-26T00:36:46.207709+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207709+0000","last_undegraded":"2024-04-26T00:36:46.207709+0000","last_fullsized":"2024-04-26T00:36:46.207709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6162,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37049,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":149037056,"data_stored":149018991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280662,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084434,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441272,"kb_used_data":88880,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93930568,"statfs":{"total":96636764160,"available":96184901632,"internally_reserved":0,"allocated":91013120,"data_stored":90710465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90562560,"data_stored":90558947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:52.542 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:53.655 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:53.655 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:53.667 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-04-26T00:36:52.590272+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37763,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496240,"kb_used_data":143436,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281619280,"statfs":{"total":289910292480,"available":288378142720,"internally_reserved":0,"allocated":146878464,"data_stored":145930738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1724,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001294"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208867+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208867+0000","last_peered":"2024-04-26T00:36:46.208867+0000","last_clean":"2024-04-26T00:36:46.208867+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208867+0000","last_undegraded":"2024-04-26T00:36:46.208867+0000","last_fullsized":"2024-04-26T00:36:46.208867+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":24332,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:52.316808+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:36:52.316808+0000","last_peered":"2024-04-26T00:36:52.316808+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:52.316808+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1436,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":99,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207709+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207709+0000","last_peered":"2024-04-26T00:36:46.207709+0000","last_clean":"2024-04-26T00:36:46.207709+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207709+0000","last_undegraded":"2024-04-26T00:36:46.207709+0000","last_fullsized":"2024-04-26T00:36:46.207709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37763,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145477632,"data_stored":145459567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280663,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084435,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437812,"kb_used_data":85420,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93934028,"statfs":{"total":96636764160,"available":96188444672,"internally_reserved":0,"allocated":87470080,"data_stored":87156580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":87003136,"data_stored":86999523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:53.669 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:54.791 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:54.791 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:54.804 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-04-26T00:36:54.590587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37763,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483752,"kb_used_data":143436,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281631768,"statfs":{"total":289910292480,"available":288390930432,"internally_reserved":0,"allocated":146878464,"data_stored":145930738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1372441708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1724,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.396386"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208867+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208867+0000","last_peered":"2024-04-26T00:36:46.208867+0000","last_clean":"2024-04-26T00:36:46.208867+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208867+0000","last_undegraded":"2024-04-26T00:36:46.208867+0000","last_fullsized":"2024-04-26T00:36:46.208867+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":24332,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:52.316808+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:36:52.316808+0000","last_peered":"2024-04-26T00:36:52.316808+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:52.316808+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1436,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37763,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145477632,"data_stored":145459567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497244,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874596,"statfs":{"total":96636764160,"available":96127586304,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":508686031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280663,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084435,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437812,"kb_used_data":85420,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93934028,"statfs":{"total":96636764160,"available":96188444672,"internally_reserved":0,"allocated":87470080,"data_stored":87156580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":87003136,"data_stored":86999523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:54.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:54.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22087, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:46.209587+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:46.209587+0000', 'last_peered': '2024-04-26T00:36:46.209587+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:46.209587+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:54.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:54.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 24332, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:52.316808+0000', 'last_change': '2024-04-26T00:36:44.574130+0000', 'last_active': '2024-04-26T00:36:52.316808+0000', 'last_peered': '2024-04-26T00:36:52.316808+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:52.316808+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2200, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1436, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2200}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:54.806 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:55.179 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:55.180 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5448/6 objects degraded (90800.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":761,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1532149760,"bytes_avail":288378142720,"bytes_total":289910292480,"degraded_objects":5448,"degraded_total":6,"degraded_ratio":908,"recovering_objects_per_sec":215,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1724,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:55.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:55.193 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:55.494 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:55.494 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:55.506 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-04-26T00:36:54.590587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37763,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1483752,"kb_used_data":143436,"kb_used_omap":12,"kb_used_meta":1340275,"kb_avail":281631768,"statfs":{"total":289910292480,"available":288390930432,"internally_reserved":0,"allocated":146878464,"data_stored":145930738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1372441708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1724,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1724,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.396386"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208867+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208867+0000","last_peered":"2024-04-26T00:36:46.208867+0000","last_clean":"2024-04-26T00:36:46.208867+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208867+0000","last_undegraded":"2024-04-26T00:36:46.208867+0000","last_fullsized":"2024-04-26T00:36:46.208867+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":24332,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:52.316808+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:36:52.316808+0000","last_peered":"2024-04-26T00:36:52.316808+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:52.316808+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2200,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1436,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5448,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37763,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":145477632,"data_stored":145459567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497244,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":496763,"kb_avail":93874596,"statfs":{"total":96636764160,"available":96127586304,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":508686031},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280663,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084435,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":437812,"kb_used_data":85420,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93934028,"statfs":{"total":96636764160,"available":96188444672,"internally_reserved":0,"allocated":87470080,"data_stored":87156580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":87003136,"data_stored":86999523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:58.509 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:58.815 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:58.815 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:58.827 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-26T00:36:58.591202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38560,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488472,"kb_used_data":139836,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281627048,"statfs":{"total":289910292480,"available":288386097152,"internally_reserved":0,"allocated":143192064,"data_stored":142244338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2521,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001908"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208867+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208867+0000","last_peered":"2024-04-26T00:36:46.208867+0000","last_clean":"2024-04-26T00:36:46.208867+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208867+0000","last_undegraded":"2024-04-26T00:36:46.208867+0000","last_fullsized":"2024-04-26T00:36:46.208867+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":25926,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:57.315464+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:36:57.315464+0000","last_peered":"2024-04-26T00:36:57.315464+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:57.315464+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38560,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141791232,"data_stored":141773167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505564,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866276,"statfs":{"total":96636764160,"available":96119066624,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280664,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084436,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434212,"kb_used_data":81820,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93937628,"statfs":{"total":96636764160,"available":96192131072,"internally_reserved":0,"allocated":83783680,"data_stored":83470180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83316736,"data_stored":83313123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:58.829 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:59.139 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:59.140 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:59.154 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-26T00:36:58.591202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38560,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488472,"kb_used_data":139836,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281627048,"statfs":{"total":289910292480,"available":288386097152,"internally_reserved":0,"allocated":143192064,"data_stored":142244338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2521,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001908"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208867+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208867+0000","last_peered":"2024-04-26T00:36:46.208867+0000","last_clean":"2024-04-26T00:36:46.208867+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208867+0000","last_undegraded":"2024-04-26T00:36:46.208867+0000","last_fullsized":"2024-04-26T00:36:46.208867+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":25926,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:57.315464+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:36:57.315464+0000","last_peered":"2024-04-26T00:36:57.315464+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:57.315464+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38560,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141791232,"data_stored":141773167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505564,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866276,"statfs":{"total":96636764160,"available":96119066624,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280664,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084436,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434212,"kb_used_data":81820,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93937628,"statfs":{"total":96636764160,"available":96192131072,"internally_reserved":0,"allocated":83783680,"data_stored":83470180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83316736,"data_stored":83313123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:36:59.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:36:59.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22087, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:46.209587+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:46.209587+0000', 'last_peered': '2024-04-26T00:36:46.209587+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:46.209587+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:59.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:36:59.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 25926, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:57.315464+0000', 'last_change': '2024-04-26T00:36:44.574130+0000', 'last_active': '2024-04-26T00:36:57.315464+0000', 'last_peered': '2024-04-26T00:36:57.315464+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:36:57.315464+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1403, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2233, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1403}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:36:59.156 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:36:59.569 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:59.570 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4651/6 objects degraded (77516.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":765,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1524195328,"bytes_avail":288386097152,"bytes_total":289910292480,"degraded_objects":4651,"degraded_total":6,"degraded_ratio":775.16666666666652,"recovering_objects_per_sec":210,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2521,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:36:59.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:36:59.583 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:36:59.880 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:36:59.881 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:36:59.894 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-04-26T00:36:58.591202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38560,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488472,"kb_used_data":139836,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281627048,"statfs":{"total":289910292480,"available":288386097152,"internally_reserved":0,"allocated":143192064,"data_stored":142244338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2521,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2521,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001908"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22021,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208867+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208867+0000","last_peered":"2024-04-26T00:36:46.208867+0000","last_clean":"2024-04-26T00:36:46.208867+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208867+0000","last_undegraded":"2024-04-26T00:36:46.208867+0000","last_fullsized":"2024-04-26T00:36:46.208867+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":25926,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:57.315464+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:36:57.315464+0000","last_peered":"2024-04-26T00:36:57.315464+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:36:57.315464+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1403,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2233,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1403}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38560,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":141791232,"data_stored":141773167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505564,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866276,"statfs":{"total":96636764160,"available":96119066624,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280664,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084436,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434212,"kb_used_data":81820,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93937628,"statfs":{"total":96636764160,"available":96192131072,"internally_reserved":0,"allocated":83783680,"data_stored":83470180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":83316736,"data_stored":83313123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:02.896 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:03.198 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:03.198 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:03.210 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-04-26T00:37:02.591941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484872,"kb_used_data":136236,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281630648,"statfs":{"total":289910292480,"available":288389783552,"internally_reserved":0,"allocated":139505664,"data_stored":138557938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2449,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2449,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001971"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208903+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208903+0000","last_peered":"2024-04-26T00:36:46.208903+0000","last_clean":"2024-04-26T00:36:46.208903+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208903+0000","last_undegraded":"2024-04-26T00:36:46.208903+0000","last_fullsized":"2024-04-26T00:36:46.208903+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":27802,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:02.316754+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:37:02.316754+0000","last_peered":"2024-04-26T00:37:02.316754+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:37:02.316754+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138104832,"data_stored":138086767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505564,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866276,"statfs":{"total":96636764160,"available":96119066624,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280665,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084437,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430612,"kb_used_data":78220,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93941228,"statfs":{"total":96636764160,"available":96195817472,"internally_reserved":0,"allocated":80097280,"data_stored":79783780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79630336,"data_stored":79626723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:03.211 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:03.531 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:03.532 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:03.544 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-04-26T00:37:02.591941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484872,"kb_used_data":136236,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281630648,"statfs":{"total":289910292480,"available":288389783552,"internally_reserved":0,"allocated":139505664,"data_stored":138557938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2449,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2449,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001971"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208903+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208903+0000","last_peered":"2024-04-26T00:36:46.208903+0000","last_clean":"2024-04-26T00:36:46.208903+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208903+0000","last_undegraded":"2024-04-26T00:36:46.208903+0000","last_fullsized":"2024-04-26T00:36:46.208903+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":27802,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:02.316754+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:37:02.316754+0000","last_peered":"2024-04-26T00:37:02.316754+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:37:02.316754+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138104832,"data_stored":138086767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505564,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866276,"statfs":{"total":96636764160,"available":96119066624,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280665,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084437,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430612,"kb_used_data":78220,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93941228,"statfs":{"total":96636764160,"available":96195817472,"internally_reserved":0,"allocated":80097280,"data_stored":79783780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79630336,"data_stored":79626723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:03.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:37:03.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22087, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:46.209587+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:46.209587+0000', 'last_peered': '2024-04-26T00:36:46.209587+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:46.209587+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:37:03.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:37:03.545 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 27802, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:37:02.316754+0000', 'last_change': '2024-04-26T00:36:44.574130+0000', 'last_active': '2024-04-26T00:37:02.316754+0000', 'last_peered': '2024-04-26T00:37:02.316754+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:37:02.316754+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 465, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3171, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -465}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:37:03.545 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:37:03.926 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:03.926 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3713/6 objects degraded (61883.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":769,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":56,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1520508928,"bytes_avail":288389783552,"bytes_total":289910292480,"degraded_objects":3713,"degraded_total":6,"degraded_ratio":618.83333333333337,"recovering_objects_per_sec":204,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2449,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:37:03.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:37:03.940 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:04.237 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:04.237 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:04.250 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-04-26T00:37:02.591941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484872,"kb_used_data":136236,"kb_used_omap":12,"kb_used_meta":1348595,"kb_avail":281630648,"statfs":{"total":289910292480,"available":288389783552,"internally_reserved":0,"allocated":139505664,"data_stored":138557938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1380961388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2449,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2449,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001971"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21804,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205575+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:36:46.205575+0000","last_peered":"2024-04-26T00:36:46.205575+0000","last_clean":"2024-04-26T00:36:46.205575+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:36:46.205575+0000","last_undegraded":"2024-04-26T00:36:46.205575+0000","last_fullsized":"2024-04-26T00:36:46.205575+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21176,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205564+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:36:46.205564+0000","last_peered":"2024-04-26T00:36:46.205564+0000","last_clean":"2024-04-26T00:36:46.205564+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:36:46.205564+0000","last_undegraded":"2024-04-26T00:36:46.205564+0000","last_fullsized":"2024-04-26T00:36:46.205564+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208903+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208903+0000","last_peered":"2024-04-26T00:36:46.208903+0000","last_clean":"2024-04-26T00:36:46.208903+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208903+0000","last_undegraded":"2024-04-26T00:36:46.208903+0000","last_fullsized":"2024-04-26T00:36:46.208903+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205923+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:36:46.205923+0000","last_peered":"2024-04-26T00:36:46.205923+0000","last_clean":"2024-04-26T00:36:46.205923+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:36:46.205923+0000","last_undegraded":"2024-04-26T00:36:46.205923+0000","last_fullsized":"2024-04-26T00:36:46.205923+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28914,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205997+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:36:46.205997+0000","last_peered":"2024-04-26T00:36:46.205997+0000","last_clean":"2024-04-26T00:36:46.205997+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:36:46.205997+0000","last_undegraded":"2024-04-26T00:36:46.205997+0000","last_fullsized":"2024-04-26T00:36:46.205997+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28348,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205139+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:36:46.205139+0000","last_peered":"2024-04-26T00:36:46.205139+0000","last_clean":"2024-04-26T00:36:46.205139+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:36:46.205139+0000","last_undegraded":"2024-04-26T00:36:46.205139+0000","last_fullsized":"2024-04-26T00:36:46.205139+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":27802,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:02.316754+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:37:02.316754+0000","last_peered":"2024-04-26T00:37:02.316754+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:37:02.316754+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.205765+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:36:46.205765+0000","last_peered":"2024-04-26T00:36:46.205765+0000","last_clean":"2024-04-26T00:36:46.205765+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:36:46.205765+0000","last_undegraded":"2024-04-26T00:36:46.205765+0000","last_fullsized":"2024-04-26T00:36:46.205765+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":138104832,"data_stored":138086767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505564,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":505083,"kb_avail":93866276,"statfs":{"total":96636764160,"available":96119066624,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":517205711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280665,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084437,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430612,"kb_used_data":78220,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93941228,"statfs":{"total":96636764160,"available":96195817472,"internally_reserved":0,"allocated":80097280,"data_stored":79783780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79630336,"data_stored":79626723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:07.252 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:07.589 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:07.589 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:07.602 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-04-26T00:37:06.592663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485444,"kb_used_data":132644,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281630076,"statfs":{"total":289910292480,"available":288389197824,"internally_reserved":0,"allocated":135827456,"data_stored":134877906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":938,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001460"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21806,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257277+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:04.257277+0000","last_peered":"2024-04-26T00:37:04.257277+0000","last_clean":"2024-04-26T00:37:04.257277+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:04.257277+0000","last_undegraded":"2024-04-26T00:37:04.257277+0000","last_fullsized":"2024-04-26T00:37:04.257277+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257406+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:04.257406+0000","last_peered":"2024-04-26T00:37:04.257406+0000","last_clean":"2024-04-26T00:37:04.257406+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:04.257406+0000","last_undegraded":"2024-04-26T00:37:04.257406+0000","last_fullsized":"2024-04-26T00:37:04.257406+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208903+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208903+0000","last_peered":"2024-04-26T00:36:46.208903+0000","last_clean":"2024-04-26T00:36:46.208903+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208903+0000","last_undegraded":"2024-04-26T00:36:46.208903+0000","last_fullsized":"2024-04-26T00:36:46.208903+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257989+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:04.257989+0000","last_peered":"2024-04-26T00:37:04.257989+0000","last_clean":"2024-04-26T00:37:04.257989+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:04.257989+0000","last_undegraded":"2024-04-26T00:37:04.257989+0000","last_fullsized":"2024-04-26T00:37:04.257989+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28916,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257893+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:04.257893+0000","last_peered":"2024-04-26T00:37:04.257893+0000","last_clean":"2024-04-26T00:37:04.257893+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:04.257893+0000","last_undegraded":"2024-04-26T00:37:04.257893+0000","last_fullsized":"2024-04-26T00:37:04.257893+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28350,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.256411+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:04.256411+0000","last_peered":"2024-04-26T00:37:04.256411+0000","last_clean":"2024-04-26T00:37:04.256411+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:04.256411+0000","last_undegraded":"2024-04-26T00:37:04.256411+0000","last_fullsized":"2024-04-26T00:37:04.256411+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":27802,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:02.316754+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:37:02.316754+0000","last_peered":"2024-04-26T00:37:02.316754+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:37:02.316754+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257586+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:04.257586+0000","last_peered":"2024-04-26T00:37:04.257586+0000","last_clean":"2024-04-26T00:37:04.257586+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:04.257586+0000","last_undegraded":"2024-04-26T00:37:04.257586+0000","last_fullsized":"2024-04-26T00:37:04.257586+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134418432,"data_stored":134403980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509724,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862116,"statfs":{"total":96636764160,"available":96114806784,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280665,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084438,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427024,"kb_used_data":74628,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93944816,"statfs":{"total":96636764160,"available":96199491584,"internally_reserved":0,"allocated":76419072,"data_stored":76103748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75943936,"data_stored":75943936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:07.604 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:08.000 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:08.000 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:08.016 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-04-26T00:37:06.592663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1485444,"kb_used_data":132644,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281630076,"statfs":{"total":289910292480,"available":288389197824,"internally_reserved":0,"allocated":135827456,"data_stored":134877906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":938,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001460"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28633,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209448+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:36:46.209448+0000","last_peered":"2024-04-26T00:36:46.209448+0000","last_clean":"2024-04-26T00:36:46.209448+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:36:46.209448+0000","last_undegraded":"2024-04-26T00:36:46.209448+0000","last_fullsized":"2024-04-26T00:36:46.209448+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21806,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257277+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:04.257277+0000","last_peered":"2024-04-26T00:37:04.257277+0000","last_clean":"2024-04-26T00:37:04.257277+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:04.257277+0000","last_undegraded":"2024-04-26T00:37:04.257277+0000","last_fullsized":"2024-04-26T00:37:04.257277+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28587,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207369+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:36:46.207369+0000","last_peered":"2024-04-26T00:36:46.207369+0000","last_clean":"2024-04-26T00:36:46.207369+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:36:46.207369+0000","last_undegraded":"2024-04-26T00:36:46.207369+0000","last_fullsized":"2024-04-26T00:36:46.207369+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257406+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:04.257406+0000","last_peered":"2024-04-26T00:37:04.257406+0000","last_clean":"2024-04-26T00:37:04.257406+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:04.257406+0000","last_undegraded":"2024-04-26T00:37:04.257406+0000","last_fullsized":"2024-04-26T00:37:04.257406+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22022,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208903+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:36:46.208903+0000","last_peered":"2024-04-26T00:36:46.208903+0000","last_clean":"2024-04-26T00:36:46.208903+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:36:46.208903+0000","last_undegraded":"2024-04-26T00:36:46.208903+0000","last_fullsized":"2024-04-26T00:36:46.208903+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22087,"reported_epoch":56,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-04-26T00:36:46.209587+0000","last_change":"2024-04-26T00:26:01.850059+0000","last_active":"2024-04-26T00:36:46.209587+0000","last_peered":"2024-04-26T00:36:46.209587+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:36:46.209587+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3248,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3246}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28335,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207113+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:36:46.207113+0000","last_peered":"2024-04-26T00:36:46.207113+0000","last_clean":"2024-04-26T00:36:46.207113+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:36:46.207113+0000","last_undegraded":"2024-04-26T00:36:46.207113+0000","last_fullsized":"2024-04-26T00:36:46.207113+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21642,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.208694+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:36:46.208694+0000","last_peered":"2024-04-26T00:36:46.208694+0000","last_clean":"2024-04-26T00:36:46.208694+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:36:46.208694+0000","last_undegraded":"2024-04-26T00:36:46.208694+0000","last_fullsized":"2024-04-26T00:36:46.208694+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207034+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:36:46.207034+0000","last_peered":"2024-04-26T00:36:46.207034+0000","last_clean":"2024-04-26T00:36:46.207034+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:36:46.207034+0000","last_undegraded":"2024-04-26T00:36:46.207034+0000","last_fullsized":"2024-04-26T00:36:46.207034+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21657,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207278+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:36:46.207278+0000","last_peered":"2024-04-26T00:36:46.207278+0000","last_clean":"2024-04-26T00:36:46.207278+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:36:46.207278+0000","last_undegraded":"2024-04-26T00:36:46.207278+0000","last_fullsized":"2024-04-26T00:36:46.207278+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:36:50.241301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257989+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:04.257989+0000","last_peered":"2024-04-26T00:37:04.257989+0000","last_clean":"2024-04-26T00:37:04.257989+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:04.257989+0000","last_undegraded":"2024-04-26T00:37:04.257989+0000","last_fullsized":"2024-04-26T00:37:04.257989+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28835,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209663+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:36:46.209663+0000","last_peered":"2024-04-26T00:36:46.209663+0000","last_clean":"2024-04-26T00:36:46.209663+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:36:46.209663+0000","last_undegraded":"2024-04-26T00:36:46.209663+0000","last_fullsized":"2024-04-26T00:36:46.209663+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":106,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.209485+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:36:46.209485+0000","last_peered":"2024-04-26T00:36:46.209485+0000","last_clean":"2024-04-26T00:36:46.209485+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:36:46.209485+0000","last_undegraded":"2024-04-26T00:36:46.209485+0000","last_fullsized":"2024-04-26T00:36:46.209485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28916,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257893+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:04.257893+0000","last_peered":"2024-04-26T00:37:04.257893+0000","last_clean":"2024-04-26T00:37:04.257893+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:04.257893+0000","last_undegraded":"2024-04-26T00:37:04.257893+0000","last_fullsized":"2024-04-26T00:37:04.257893+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207193+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:36:46.207193+0000","last_peered":"2024-04-26T00:36:46.207193+0000","last_clean":"2024-04-26T00:36:46.207193+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:36:46.207193+0000","last_undegraded":"2024-04-26T00:36:46.207193+0000","last_fullsized":"2024-04-26T00:36:46.207193+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28350,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.256411+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:04.256411+0000","last_peered":"2024-04-26T00:37:04.256411+0000","last_clean":"2024-04-26T00:37:04.256411+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:04.256411+0000","last_undegraded":"2024-04-26T00:37:04.256411+0000","last_fullsized":"2024-04-26T00:37:04.256411+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":27802,"reported_epoch":56,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:02.316754+0000","last_change":"2024-04-26T00:36:44.574130+0000","last_active":"2024-04-26T00:37:02.316754+0000","last_peered":"2024-04-26T00:37:02.316754+0000","last_clean":"2024-04-26T00:25:59.336399+0000","last_became_active":"2024-04-26T00:26:01.840069+0000","last_became_peered":"2024-04-26T00:26:01.840069+0000","last_unstale":"2024-04-26T00:37:02.316754+0000","last_undegraded":"2024-04-26T00:26:01.454398+0000","last_fullsized":"2024-04-26T00:26:01.360905+0000","mapping_epoch":35,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":465,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-465}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207319+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:36:46.207319+0000","last_peered":"2024-04-26T00:36:46.207319+0000","last_clean":"2024-04-26T00:36:46.207319+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:36:46.207319+0000","last_undegraded":"2024-04-26T00:36:46.207319+0000","last_fullsized":"2024-04-26T00:36:46.207319+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21823,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.281217+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:36:46.281217+0000","last_peered":"2024-04-26T00:36:46.281217+0000","last_clean":"2024-04-26T00:36:46.281217+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:36:46.281217+0000","last_undegraded":"2024-04-26T00:36:46.281217+0000","last_fullsized":"2024-04-26T00:36:46.281217+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257586+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:04.257586+0000","last_peered":"2024-04-26T00:37:04.257586+0000","last_clean":"2024-04-26T00:37:04.257586+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:04.257586+0000","last_undegraded":"2024-04-26T00:37:04.257586+0000","last_fullsized":"2024-04-26T00:37:04.257586+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22155,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207512+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:36:46.207512+0000","last_peered":"2024-04-26T00:36:46.207512+0000","last_clean":"2024-04-26T00:36:46.207512+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:36:46.207512+0000","last_undegraded":"2024-04-26T00:36:46.207512+0000","last_fullsized":"2024-04-26T00:36:46.207512+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":96,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207133+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:36:46.207133+0000","last_peered":"2024-04-26T00:36:46.207133+0000","last_clean":"2024-04-26T00:36:46.207133+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:36:46.207133+0000","last_undegraded":"2024-04-26T00:36:46.207133+0000","last_fullsized":"2024-04-26T00:36:46.207133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":100,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207747+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:36:46.207747+0000","last_peered":"2024-04-26T00:36:46.207747+0000","last_clean":"2024-04-26T00:36:46.207747+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:36:46.207747+0000","last_undegraded":"2024-04-26T00:36:46.207747+0000","last_fullsized":"2024-04-26T00:36:46.207747+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28763,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-04-26T00:36:46.207318+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:36:46.207318+0000","last_peered":"2024-04-26T00:36:46.207318+0000","last_clean":"2024-04-26T00:36:46.207318+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:36:46.207318+0000","last_undegraded":"2024-04-26T00:36:46.207318+0000","last_fullsized":"2024-04-26T00:36:46.207318+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3713,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39498,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134418432,"data_stored":134403980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509724,"kb_used_data":468,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862116,"statfs":{"total":96636764160,"available":96114806784,"internally_reserved":0,"allocated":479232,"data_stored":158506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280665,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548696,"kb_used_data":57548,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823144,"statfs":{"total":96636764160,"available":96074899456,"internally_reserved":0,"allocated":58929152,"data_stored":58615652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084438,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427024,"kb_used_data":74628,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93944816,"statfs":{"total":96636764160,"available":96199491584,"internally_reserved":0,"allocated":76419072,"data_stored":76103748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75943936,"data_stored":75943936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:08.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:37:08.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22087, 'reported_epoch': 56, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:36:46.209587+0000', 'last_change': '2024-04-26T00:26:01.850059+0000', 'last_active': '2024-04-26T00:36:46.209587+0000', 'last_peered': '2024-04-26T00:36:46.209587+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:36:46.209587+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3248, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 327, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3246}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:37:08.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-04-26T00:37:08.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "44'21304", 'reported_seq': 27802, 'reported_epoch': 56, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:37:02.316754+0000', 'last_change': '2024-04-26T00:36:44.574130+0000', 'last_active': '2024-04-26T00:37:02.316754+0000', 'last_peered': '2024-04-26T00:37:02.316754+0000', 'last_clean': '2024-04-26T00:25:59.336399+0000', 'last_became_active': '2024-04-26T00:26:01.840069+0000', 'last_became_peered': '2024-04-26T00:26:01.840069+0000', 'last_unstale': '2024-04-26T00:37:02.316754+0000', 'last_undegraded': '2024-04-26T00:26:01.454398+0000', 'last_fullsized': '2024-04-26T00:26:01.360905+0000', 'mapping_epoch': 35, 'log_start': "38'11300", 'ondisk_log_start': "38'11300", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'3099", 'last_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:49.772261+0000', 'objects_scrubbed': 3077, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 803, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 465, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18225, 'num_write_kb': 30296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3171, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -465}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:37:08.017 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:37:08.396 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:08.396 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3713/6 objects degraded (61883.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1521094656,"bytes_avail":288389197824,"bytes_total":289910292480,"degraded_objects":3713,"degraded_total":6,"degraded_ratio":618.83333333333337,"recovering_objects_per_sec":117,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":938,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:37:08.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:37:08.409 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:08.719 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:08.719 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:08.732 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-04-26T00:37:08.592958+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40397,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477152,"kb_used_data":132668,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281638368,"statfs":{"total":289910292480,"available":288397688832,"internally_reserved":0,"allocated":135852032,"data_stored":134887918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1837,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1837,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001357"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21806,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257277+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:04.257277+0000","last_peered":"2024-04-26T00:37:04.257277+0000","last_clean":"2024-04-26T00:37:04.257277+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:04.257277+0000","last_undegraded":"2024-04-26T00:37:04.257277+0000","last_fullsized":"2024-04-26T00:37:04.257277+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21178,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257406+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:04.257406+0000","last_peered":"2024-04-26T00:37:04.257406+0000","last_clean":"2024-04-26T00:37:04.257406+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:04.257406+0000","last_undegraded":"2024-04-26T00:37:04.257406+0000","last_fullsized":"2024-04-26T00:37:04.257406+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22960,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:07.310662+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:07.310662+0000","last_peered":"2024-04-26T00:37:07.310662+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:07.310662+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257989+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:04.257989+0000","last_peered":"2024-04-26T00:37:04.257989+0000","last_clean":"2024-04-26T00:37:04.257989+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:04.257989+0000","last_undegraded":"2024-04-26T00:37:04.257989+0000","last_fullsized":"2024-04-26T00:37:04.257989+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28916,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257893+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:04.257893+0000","last_peered":"2024-04-26T00:37:04.257893+0000","last_clean":"2024-04-26T00:37:04.257893+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:04.257893+0000","last_undegraded":"2024-04-26T00:37:04.257893+0000","last_fullsized":"2024-04-26T00:37:04.257893+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28350,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.256411+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:04.256411+0000","last_peered":"2024-04-26T00:37:04.256411+0000","last_clean":"2024-04-26T00:37:04.256411+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:04.256411+0000","last_undegraded":"2024-04-26T00:37:04.256411+0000","last_fullsized":"2024-04-26T00:37:04.256411+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-04-26T00:37:04.257586+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:04.257586+0000","last_peered":"2024-04-26T00:37:04.257586+0000","last_clean":"2024-04-26T00:37:04.257586+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:04.257586+0000","last_undegraded":"2024-04-26T00:37:04.257586+0000","last_fullsized":"2024-04-26T00:37:04.257586+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40397,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":134410240,"data_stored":134403014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280666,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084438,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427024,"kb_used_data":74628,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93944816,"statfs":{"total":96636764160,"available":96199491584,"internally_reserved":0,"allocated":76419072,"data_stored":76103748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75943936,"data_stored":75943936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:11.734 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:12.019 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:12.019 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:12.031 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-04-26T00:37:10.593297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40397,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472960,"kb_used_data":128480,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281642560,"statfs":{"total":289910292480,"available":288401981440,"internally_reserved":0,"allocated":131563520,"data_stored":130593948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":899,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001355"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22960,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:07.310662+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:07.310662+0000","last_peered":"2024-04-26T00:37:07.310662+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:07.310662+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40397,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130113536,"data_stored":130106310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280666,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084439,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422832,"kb_used_data":70440,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93949008,"statfs":{"total":96636764160,"available":96203784192,"internally_reserved":0,"allocated":72130560,"data_stored":71809778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71647232,"data_stored":71647232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:12.032 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:12.354 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:12.354 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:12.366 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-04-26T00:37:10.593297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40397,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472960,"kb_used_data":128480,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281642560,"statfs":{"total":289910292480,"available":288401981440,"internally_reserved":0,"allocated":131563520,"data_stored":130593948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-899,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":899,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001355"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22960,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:07.310662+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:07.310662+0000","last_peered":"2024-04-26T00:37:07.310662+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:07.310662+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":761,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2814}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2814,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40397,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130113536,"data_stored":130106310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280666,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084439,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422832,"kb_used_data":70440,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93949008,"statfs":{"total":96636764160,"available":96203784192,"internally_reserved":0,"allocated":72130560,"data_stored":71809778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71647232,"data_stored":71647232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:12.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:37:12.368 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 22960, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:37:07.310662+0000', 'last_change': '2024-04-26T00:37:04.133752+0000', 'last_active': '2024-04-26T00:37:07.310662+0000', 'last_peered': '2024-04-26T00:37:07.310662+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:37:07.310662+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2814, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 761, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2814}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:37:12.368 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:37:12.742 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:12.742 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2814/6 objects degraded (46900.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":778,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1508311040,"bytes_avail":288401981440,"bytes_total":289910292480,"degraded_objects":2814,"degraded_total":6,"degraded_ratio":469,"recovering_objects_per_sec":112,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":899,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:37:12.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:37:12.755 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:13.065 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:13.066 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:13.078 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-04-26T00:37:12.593646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41046,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472960,"kb_used_data":128480,"kb_used_omap":12,"kb_used_meta":1344435,"kb_avail":281642560,"statfs":{"total":289910292480,"available":288401981440,"internally_reserved":0,"allocated":131563520,"data_stored":130593948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1376701548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1548,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.338115"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":24258,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:12.317074+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:12.317074+0000","last_peered":"2024-04-26T00:37:12.317074+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:12.317074+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41046,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130113536,"data_stored":130106310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501412,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":500923,"kb_avail":93870428,"statfs":{"total":96636764160,"available":96123318272,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":512945871},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280667,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084439,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422832,"kb_used_data":70440,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93949008,"statfs":{"total":96636764160,"available":96203784192,"internally_reserved":0,"allocated":72130560,"data_stored":71809778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71647232,"data_stored":71647232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:16.080 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:16.387 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:16.387 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:16.399 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":442,"stamp":"2024-04-26T00:37:14.593922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41046,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481280,"kb_used_data":128480,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281634240,"statfs":{"total":289910292480,"available":288393461760,"internally_reserved":0,"allocated":131563520,"data_stored":130593948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1548,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001610"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":24258,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:12.317074+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:12.317074+0000","last_peered":"2024-04-26T00:37:12.317074+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:12.317074+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41046,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":130113536,"data_stored":130106310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280667,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084439,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422832,"kb_used_data":70440,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93949008,"statfs":{"total":96636764160,"available":96203784192,"internally_reserved":0,"allocated":72130560,"data_stored":71809778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":71647232,"data_stored":71647232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:16.400 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:16.714 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:16.714 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:16.727 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-04-26T00:37:16.594210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41046,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477080,"kb_used_data":124280,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281638440,"statfs":{"total":289910292480,"available":288397762560,"internally_reserved":0,"allocated":127262720,"data_stored":126293148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1548,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.340536"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":24258,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:12.317074+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:12.317074+0000","last_peered":"2024-04-26T00:37:12.317074+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:12.317074+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41046,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125812736,"data_stored":125805510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280667,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084440,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418632,"kb_used_data":66240,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93953208,"statfs":{"total":96636764160,"available":96208084992,"internally_reserved":0,"allocated":67829760,"data_stored":67508978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":67346432,"data_stored":67346432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:16.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:37:16.728 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 24258, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:37:12.317074+0000', 'last_change': '2024-04-26T00:37:04.133752+0000', 'last_active': '2024-04-26T00:37:12.317074+0000', 'last_peered': '2024-04-26T00:37:12.317074+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:37:12.317074+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2165, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1410, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2165}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:37:16.728 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:37:17.111 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:17.111 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2165/6 objects degraded (36083.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":783,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1516830720,"bytes_avail":288393461760,"bytes_total":289910292480,"degraded_objects":2165,"degraded_total":6,"degraded_ratio":360.83333333333326,"recovering_objects_per_sec":154,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1548,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:37:17.123 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:37:17.124 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:17.476 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:17.476 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:17.489 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-04-26T00:37:16.594210+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41046,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477080,"kb_used_data":124280,"kb_used_omap":12,"kb_used_meta":1352755,"kb_avail":281638440,"statfs":{"total":289910292480,"available":288397762560,"internally_reserved":0,"allocated":127262720,"data_stored":126293148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1385221228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1548,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1548,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.340536"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":24258,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:12.317074+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:12.317074+0000","last_peered":"2024-04-26T00:37:12.317074+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:12.317074+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2165}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2165,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41046,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125812736,"data_stored":125805510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509732,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":509243,"kb_avail":93862108,"statfs":{"total":96636764160,"available":96114798592,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":521465551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280667,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084440,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418632,"kb_used_data":66240,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93953208,"statfs":{"total":96636764160,"available":96208084992,"internally_reserved":0,"allocated":67829760,"data_stored":67508978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":67346432,"data_stored":67346432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:20.491 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:21.599 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:21.600 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:21.611 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-04-26T00:37:20.594811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478448,"kb_used_data":120080,"kb_used_omap":12,"kb_used_meta":1358323,"kb_avail":281637072,"statfs":{"total":289910292480,"available":288396361728,"internally_reserved":0,"allocated":122961920,"data_stored":121992348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1390922860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001853"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":25780,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:17.319136+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:17.319136+0000","last_peered":"2024-04-26T00:37:17.319136+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:17.319136+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":121511936,"data_stored":121504710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515300,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856540,"statfs":{"total":96636764160,"available":96109096960,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":527167183},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280668,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084441,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414432,"kb_used_data":62040,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93957408,"statfs":{"total":96636764160,"available":96212385792,"internally_reserved":0,"allocated":63528960,"data_stored":63208178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":63045632,"data_stored":63045632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:21.613 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:21.929 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:21.930 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:21.942 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-04-26T00:37:20.594811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478448,"kb_used_data":120080,"kb_used_omap":12,"kb_used_meta":1358323,"kb_avail":281637072,"statfs":{"total":289910292480,"available":288396361728,"internally_reserved":0,"allocated":122961920,"data_stored":121992348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1390922860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001853"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":25780,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:17.319136+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:17.319136+0000","last_peered":"2024-04-26T00:37:17.319136+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:17.319136+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41807,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":121511936,"data_stored":121504710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515300,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856540,"statfs":{"total":96636764160,"available":96109096960,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":527167183},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280668,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084441,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414432,"kb_used_data":62040,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93957408,"statfs":{"total":96636764160,"available":96212385792,"internally_reserved":0,"allocated":63528960,"data_stored":63208178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":63045632,"data_stored":63045632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:21.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:37:21.944 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 25780, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:37:17.319136+0000', 'last_change': '2024-04-26T00:37:04.133752+0000', 'last_active': '2024-04-26T00:37:17.319136+0000', 'last_peered': '2024-04-26T00:37:17.319136+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:37:17.319136+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1404, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2171, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1404}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:37:21.944 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:37:22.314 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:22.315 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1404/6 objects degraded (23400.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":788,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":58,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1513930752,"bytes_avail":288396361728,"bytes_total":289910292480,"degraded_objects":1404,"degraded_total":6,"degraded_ratio":234,"recovering_objects_per_sec":117,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1410,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:37:22.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:37:22.329 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:22.638 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:22.639 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:22.652 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2024-04-26T00:37:22.595143+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42804,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478448,"kb_used_data":120080,"kb_used_omap":12,"kb_used_meta":1358323,"kb_avail":281637072,"statfs":{"total":289910292480,"available":288396361728,"internally_reserved":0,"allocated":122961920,"data_stored":121992348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1390922860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2407,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001845"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21808,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254601+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:05.254601+0000","last_peered":"2024-04-26T00:37:05.254601+0000","last_clean":"2024-04-26T00:37:05.254601+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:05.254601+0000","last_undegraded":"2024-04-26T00:37:05.254601+0000","last_fullsized":"2024-04-26T00:37:05.254601+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21180,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254130+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:05.254130+0000","last_peered":"2024-04-26T00:37:05.254130+0000","last_clean":"2024-04-26T00:37:05.254130+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:05.254130+0000","last_undegraded":"2024-04-26T00:37:05.254130+0000","last_fullsized":"2024-04-26T00:37:05.254130+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":27774,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:22.319397+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:22.319397+0000","last_peered":"2024-04-26T00:37:22.319397+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:22.319397+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254192+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:05.254192+0000","last_peered":"2024-04-26T00:37:05.254192+0000","last_clean":"2024-04-26T00:37:05.254192+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:05.254192+0000","last_undegraded":"2024-04-26T00:37:05.254192+0000","last_fullsized":"2024-04-26T00:37:05.254192+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28918,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254269+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:05.254269+0000","last_peered":"2024-04-26T00:37:05.254269+0000","last_clean":"2024-04-26T00:37:05.254269+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:05.254269+0000","last_undegraded":"2024-04-26T00:37:05.254269+0000","last_fullsized":"2024-04-26T00:37:05.254269+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28353,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254059+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:05.254059+0000","last_peered":"2024-04-26T00:37:05.254059+0000","last_clean":"2024-04-26T00:37:05.254059+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:05.254059+0000","last_undegraded":"2024-04-26T00:37:05.254059+0000","last_fullsized":"2024-04-26T00:37:05.254059+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254338+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:05.254338+0000","last_peered":"2024-04-26T00:37:05.254338+0000","last_clean":"2024-04-26T00:37:05.254338+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:05.254338+0000","last_undegraded":"2024-04-26T00:37:05.254338+0000","last_fullsized":"2024-04-26T00:37:05.254338+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42804,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":121511936,"data_stored":121504710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515300,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":514811,"kb_avail":93856540,"statfs":{"total":96636764160,"available":96109096960,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":527167183},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280669,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084441,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414432,"kb_used_data":62040,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93957408,"statfs":{"total":96636764160,"available":96212385792,"internally_reserved":0,"allocated":63528960,"data_stored":63208178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":63045632,"data_stored":63045632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:25.655 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:25.941 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:25.941 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:25.953 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-04-26T00:37:25.333117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42804,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482580,"kb_used_data":115888,"kb_used_omap":12,"kb_used_meta":1366643,"kb_avail":281632940,"statfs":{"total":289910292480,"available":288392130560,"internally_reserved":0,"allocated":118669312,"data_stored":117694278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1399442540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1758,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.738907"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327947+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:24.327947+0000","last_peered":"2024-04-26T00:37:24.327947+0000","last_clean":"2024-04-26T00:37:24.327947+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:24.327947+0000","last_undegraded":"2024-04-26T00:37:24.327947+0000","last_fullsized":"2024-04-26T00:37:24.327947+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21182,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326938+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:24.326938+0000","last_peered":"2024-04-26T00:37:24.326938+0000","last_clean":"2024-04-26T00:37:24.326938+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:24.326938+0000","last_undegraded":"2024-04-26T00:37:24.326938+0000","last_fullsized":"2024-04-26T00:37:24.326938+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":27774,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:22.319397+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:22.319397+0000","last_peered":"2024-04-26T00:37:22.319397+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:22.319397+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326712+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:24.326712+0000","last_peered":"2024-04-26T00:37:24.326712+0000","last_clean":"2024-04-26T00:37:24.326712+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:24.326712+0000","last_undegraded":"2024-04-26T00:37:24.326712+0000","last_fullsized":"2024-04-26T00:37:24.326712+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28920,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326794+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:24.326794+0000","last_peered":"2024-04-26T00:37:24.326794+0000","last_clean":"2024-04-26T00:37:24.326794+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:24.326794+0000","last_undegraded":"2024-04-26T00:37:24.326794+0000","last_fullsized":"2024-04-26T00:37:24.326794+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28355,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326577+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:24.326577+0000","last_peered":"2024-04-26T00:37:24.326577+0000","last_clean":"2024-04-26T00:37:24.326577+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:24.326577+0000","last_undegraded":"2024-04-26T00:37:24.326577+0000","last_fullsized":"2024-04-26T00:37:24.326577+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327058+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:24.327058+0000","last_peered":"2024-04-26T00:37:24.327058+0000","last_clean":"2024-04-26T00:37:24.327058+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:24.327058+0000","last_undegraded":"2024-04-26T00:37:24.327058+0000","last_fullsized":"2024-04-26T00:37:24.327058+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42804,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117211136,"data_stored":117203910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523620,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848220,"statfs":{"total":96636764160,"available":96100577280,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":535686863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280669,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084442,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410244,"kb_used_data":57848,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961596,"statfs":{"total":96636764160,"available":96216674304,"internally_reserved":0,"allocated":59236352,"data_stored":58910108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58744832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:25.954 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:26.232 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:26.232 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:26.244 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":449,"stamp":"2024-04-26T00:37:25.333117+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42804,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482580,"kb_used_data":115888,"kb_used_omap":12,"kb_used_meta":1366643,"kb_avail":281632940,"statfs":{"total":289910292480,"available":288392130560,"internally_reserved":0,"allocated":118669312,"data_stored":117694278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1399442540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1758,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.738907"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327947+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:24.327947+0000","last_peered":"2024-04-26T00:37:24.327947+0000","last_clean":"2024-04-26T00:37:24.327947+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:24.327947+0000","last_undegraded":"2024-04-26T00:37:24.327947+0000","last_fullsized":"2024-04-26T00:37:24.327947+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21182,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326938+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:24.326938+0000","last_peered":"2024-04-26T00:37:24.326938+0000","last_clean":"2024-04-26T00:37:24.326938+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:24.326938+0000","last_undegraded":"2024-04-26T00:37:24.326938+0000","last_fullsized":"2024-04-26T00:37:24.326938+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":27774,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:22.319397+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:22.319397+0000","last_peered":"2024-04-26T00:37:22.319397+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:22.319397+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326712+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:24.326712+0000","last_peered":"2024-04-26T00:37:24.326712+0000","last_clean":"2024-04-26T00:37:24.326712+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:24.326712+0000","last_undegraded":"2024-04-26T00:37:24.326712+0000","last_fullsized":"2024-04-26T00:37:24.326712+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28920,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326794+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:24.326794+0000","last_peered":"2024-04-26T00:37:24.326794+0000","last_clean":"2024-04-26T00:37:24.326794+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:24.326794+0000","last_undegraded":"2024-04-26T00:37:24.326794+0000","last_fullsized":"2024-04-26T00:37:24.326794+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28355,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326577+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:24.326577+0000","last_peered":"2024-04-26T00:37:24.326577+0000","last_clean":"2024-04-26T00:37:24.326577+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:24.326577+0000","last_undegraded":"2024-04-26T00:37:24.326577+0000","last_fullsized":"2024-04-26T00:37:24.326577+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327058+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:24.327058+0000","last_peered":"2024-04-26T00:37:24.327058+0000","last_clean":"2024-04-26T00:37:24.327058+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:24.327058+0000","last_undegraded":"2024-04-26T00:37:24.327058+0000","last_fullsized":"2024-04-26T00:37:24.327058+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42804,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117211136,"data_stored":117203910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523620,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848220,"statfs":{"total":96636764160,"available":96100577280,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":535686863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280669,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084442,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410244,"kb_used_data":57848,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961596,"statfs":{"total":96636764160,"available":96216674304,"internally_reserved":0,"allocated":59236352,"data_stored":58910108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58744832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-04-26T00:37:26.245 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "44'21275", 'reported_seq': 27774, 'reported_epoch': 58, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-04-26T00:37:22.319397+0000', 'last_change': '2024-04-26T00:37:04.133752+0000', 'last_active': '2024-04-26T00:37:22.319397+0000', 'last_peered': '2024-04-26T00:37:22.319397+0000', 'last_clean': '2024-04-26T00:25:59.337250+0000', 'last_became_active': '2024-04-26T00:26:01.834061+0000', 'last_became_peered': '2024-04-26T00:26:01.834061+0000', 'last_unstale': '2024-04-26T00:37:22.319397+0000', 'last_undegraded': '2024-04-26T00:26:01.428054+0000', 'last_fullsized': '2024-04-26T00:26:01.362291+0000', 'mapping_epoch': 35, 'log_start': "38'11200", 'ondisk_log_start': "38'11200", 'created': 13, 'last_epoch_clean': 32, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "29'2845", 'last_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-26T00:24:32.507794+0000', 'last_clean_scrub_stamp': '2024-04-26T00:25:46.819941+0000', 'objects_scrubbed': 2820, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 941, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 407, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18450, 'num_write_kb': 31260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3168, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -407}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-04-26T00:37:26.246 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-26T00:37:26.594 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:26.594 INFO:teuthology.orchestra.run.smithi072.stdout:{"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 407/6 objects degraded (6783.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":792,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":3,"num_up_osds":3,"osd_up_since":1714091159,"num_in_osds":3,"osd_in_since":1714091153,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":1449,"bytes_used":1522450432,"bytes_avail":288387842048,"bytes_total":289910292480,"degraded_objects":407,"degraded_total":6,"degraded_ratio":67.833333333333329,"recovering_objects_per_sec":175,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1758,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-04-26T00:36:24.585830+0000","services":{}},"progress_events":{}} 2024-04-26T00:37:26.607 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-04-26T00:37:26.608 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:26.885 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:26.885 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:26.897 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":450,"stamp":"2024-04-26T00:37:26.596049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42804,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1482580,"kb_used_data":115888,"kb_used_omap":12,"kb_used_meta":1366643,"kb_avail":281632940,"statfs":{"total":289910292480,"available":288392130560,"internally_reserved":0,"allocated":118669312,"data_stored":117694278,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1399442540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":997,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001553"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28637,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255135+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:05.255135+0000","last_peered":"2024-04-26T00:37:05.255135+0000","last_clean":"2024-04-26T00:37:05.255135+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:05.255135+0000","last_undegraded":"2024-04-26T00:37:05.255135+0000","last_fullsized":"2024-04-26T00:37:05.255135+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327947+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:24.327947+0000","last_peered":"2024-04-26T00:37:24.327947+0000","last_clean":"2024-04-26T00:37:24.327947+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:24.327947+0000","last_undegraded":"2024-04-26T00:37:24.327947+0000","last_fullsized":"2024-04-26T00:37:24.327947+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28591,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256149+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:05.256149+0000","last_peered":"2024-04-26T00:37:05.256149+0000","last_clean":"2024-04-26T00:37:05.256149+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:05.256149+0000","last_undegraded":"2024-04-26T00:37:05.256149+0000","last_fullsized":"2024-04-26T00:37:05.256149+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21182,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326938+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:24.326938+0000","last_peered":"2024-04-26T00:37:24.326938+0000","last_clean":"2024-04-26T00:37:24.326938+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:24.326938+0000","last_undegraded":"2024-04-26T00:37:24.326938+0000","last_fullsized":"2024-04-26T00:37:24.326938+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22026,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255173+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:05.255173+0000","last_peered":"2024-04-26T00:37:05.255173+0000","last_clean":"2024-04-26T00:37:05.255173+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:05.255173+0000","last_undegraded":"2024-04-26T00:37:05.255173+0000","last_fullsized":"2024-04-26T00:37:05.255173+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":27774,"reported_epoch":58,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-04-26T00:37:22.319397+0000","last_change":"2024-04-26T00:37:04.133752+0000","last_active":"2024-04-26T00:37:22.319397+0000","last_peered":"2024-04-26T00:37:22.319397+0000","last_clean":"2024-04-26T00:25:59.337250+0000","last_became_active":"2024-04-26T00:26:01.834061+0000","last_became_peered":"2024-04-26T00:26:01.834061+0000","last_unstale":"2024-04-26T00:37:22.319397+0000","last_undegraded":"2024-04-26T00:26:01.428054+0000","last_fullsized":"2024-04-26T00:26:01.362291+0000","mapping_epoch":35,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3168,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-407}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28339,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256066+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:05.256066+0000","last_peered":"2024-04-26T00:37:05.256066+0000","last_clean":"2024-04-26T00:37:05.256066+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:05.256066+0000","last_undegraded":"2024-04-26T00:37:05.256066+0000","last_fullsized":"2024-04-26T00:37:05.256066+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21646,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.254776+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:05.254776+0000","last_peered":"2024-04-26T00:37:05.254776+0000","last_clean":"2024-04-26T00:37:05.254776+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:05.254776+0000","last_undegraded":"2024-04-26T00:37:05.254776+0000","last_fullsized":"2024-04-26T00:37:05.254776+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255923+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:05.255923+0000","last_peered":"2024-04-26T00:37:05.255923+0000","last_clean":"2024-04-26T00:37:05.255923+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:05.255923+0000","last_undegraded":"2024-04-26T00:37:05.255923+0000","last_fullsized":"2024-04-26T00:37:05.255923+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21661,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256073+0000","last_change":"2024-04-26T00:35:50.098814+0000","last_active":"2024-04-26T00:37:05.256073+0000","last_peered":"2024-04-26T00:37:05.256073+0000","last_clean":"2024-04-26T00:37:05.256073+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:05.256073+0000","last_undegraded":"2024-04-26T00:37:05.256073+0000","last_fullsized":"2024-04-26T00:37:05.256073+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":1024,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:37:06.320721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":966,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20693,"num_write_kb":41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3467,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326712+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:24.326712+0000","last_peered":"2024-04-26T00:37:24.326712+0000","last_clean":"2024-04-26T00:37:24.326712+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:24.326712+0000","last_undegraded":"2024-04-26T00:37:24.326712+0000","last_fullsized":"2024-04-26T00:37:24.326712+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28839,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255142+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:05.255142+0000","last_peered":"2024-04-26T00:37:05.255142+0000","last_clean":"2024-04-26T00:37:05.255142+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:05.255142+0000","last_undegraded":"2024-04-26T00:37:05.255142+0000","last_fullsized":"2024-04-26T00:37:05.255142+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":110,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255030+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:05.255030+0000","last_peered":"2024-04-26T00:37:05.255030+0000","last_clean":"2024-04-26T00:37:05.255030+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:05.255030+0000","last_undegraded":"2024-04-26T00:37:05.255030+0000","last_fullsized":"2024-04-26T00:37:05.255030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28920,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326794+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:24.326794+0000","last_peered":"2024-04-26T00:37:24.326794+0000","last_clean":"2024-04-26T00:37:24.326794+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:24.326794+0000","last_undegraded":"2024-04-26T00:37:24.326794+0000","last_fullsized":"2024-04-26T00:37:24.326794+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.255985+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:05.255985+0000","last_peered":"2024-04-26T00:37:05.255985+0000","last_clean":"2024-04-26T00:37:05.255985+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:05.255985+0000","last_undegraded":"2024-04-26T00:37:05.255985+0000","last_fullsized":"2024-04-26T00:37:05.255985+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28355,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326577+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:24.326577+0000","last_peered":"2024-04-26T00:37:24.326577+0000","last_clean":"2024-04-26T00:37:24.326577+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:24.326577+0000","last_undegraded":"2024-04-26T00:37:24.326577+0000","last_fullsized":"2024-04-26T00:37:24.326577+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22046,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.335850+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:05.335850+0000","last_peered":"2024-04-26T00:37:05.335850+0000","last_clean":"2024-04-26T00:37:05.335850+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:05.335850+0000","last_undegraded":"2024-04-26T00:37:05.335850+0000","last_fullsized":"2024-04-26T00:37:05.335850+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256125+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:05.256125+0000","last_peered":"2024-04-26T00:37:05.256125+0000","last_clean":"2024-04-26T00:37:05.256125+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:05.256125+0000","last_undegraded":"2024-04-26T00:37:05.256125+0000","last_fullsized":"2024-04-26T00:37:05.256125+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21827,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256236+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:05.256236+0000","last_peered":"2024-04-26T00:37:05.256236+0000","last_clean":"2024-04-26T00:37:05.256236+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:05.256236+0000","last_undegraded":"2024-04-26T00:37:05.256236+0000","last_fullsized":"2024-04-26T00:37:05.256236+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327058+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:24.327058+0000","last_peered":"2024-04-26T00:37:24.327058+0000","last_clean":"2024-04-26T00:37:24.327058+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:24.327058+0000","last_undegraded":"2024-04-26T00:37:24.327058+0000","last_fullsized":"2024-04-26T00:37:24.327058+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22159,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":100,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256239+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:05.256239+0000","last_peered":"2024-04-26T00:37:05.256239+0000","last_clean":"2024-04-26T00:37:05.256239+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:05.256239+0000","last_undegraded":"2024-04-26T00:37:05.256239+0000","last_fullsized":"2024-04-26T00:37:05.256239+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":104,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256312+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:05.256312+0000","last_peered":"2024-04-26T00:37:05.256312+0000","last_clean":"2024-04-26T00:37:05.256312+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:05.256312+0000","last_undegraded":"2024-04-26T00:37:05.256312+0000","last_fullsized":"2024-04-26T00:37:05.256312+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28767,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-04-26T00:37:05.256005+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:05.256005+0000","last_peered":"2024-04-26T00:37:05.256005+0000","last_clean":"2024-04-26T00:37:05.256005+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:05.256005+0000","last_undegraded":"2024-04-26T00:37:05.256005+0000","last_fullsized":"2024-04-26T00:37:05.256005+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1449,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":301175,"num_write_kb":524341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42804,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117211136,"data_stored":117203910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523620,"kb_used_data":476,"kb_used_omap":4,"kb_used_meta":523131,"kb_avail":93848220,"statfs":{"total":96636764160,"available":96100577280,"internally_reserved":0,"allocated":487424,"data_stored":163029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":535686863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280669,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548716,"kb_used_data":57564,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93823124,"statfs":{"total":96636764160,"available":96074878976,"internally_reserved":0,"allocated":58945536,"data_stored":58621141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084442,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410244,"kb_used_data":57848,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961596,"statfs":{"total":96636764160,"available":96216674304,"internally_reserved":0,"allocated":59236352,"data_stored":58910108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":58462208,"data_stored":58458595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58744832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:29.899 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:30.189 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:30.189 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:30.201 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-04-26T00:37:28.596407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484872,"kb_used_data":114016,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281630648,"statfs":{"total":289910292480,"available":288389783552,"internally_reserved":0,"allocated":116752384,"data_stored":115755460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1403702380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20693,"num_write_kb":-41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2063,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001596"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28641,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337606+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:25.337606+0000","last_peered":"2024-04-26T00:37:25.337606+0000","last_clean":"2024-04-26T00:37:25.337606+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:25.337606+0000","last_undegraded":"2024-04-26T00:37:25.337606+0000","last_fullsized":"2024-04-26T00:37:25.337606+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327947+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:24.327947+0000","last_peered":"2024-04-26T00:37:24.327947+0000","last_clean":"2024-04-26T00:37:24.327947+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:24.327947+0000","last_undegraded":"2024-04-26T00:37:24.327947+0000","last_fullsized":"2024-04-26T00:37:24.327947+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28595,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337182+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:25.337182+0000","last_peered":"2024-04-26T00:37:25.337182+0000","last_clean":"2024-04-26T00:37:25.337182+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:25.337182+0000","last_undegraded":"2024-04-26T00:37:25.337182+0000","last_fullsized":"2024-04-26T00:37:25.337182+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21182,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326938+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:24.326938+0000","last_peered":"2024-04-26T00:37:24.326938+0000","last_clean":"2024-04-26T00:37:24.326938+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:24.326938+0000","last_undegraded":"2024-04-26T00:37:24.326938+0000","last_fullsized":"2024-04-26T00:37:24.326938+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22030,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337303+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:25.337303+0000","last_peered":"2024-04-26T00:37:25.337303+0000","last_clean":"2024-04-26T00:37:25.337303+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:25.337303+0000","last_undegraded":"2024-04-26T00:37:25.337303+0000","last_fullsized":"2024-04-26T00:37:25.337303+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.452527+0000","last_change":"2024-04-26T00:37:25.452038+0000","last_active":"2024-04-26T00:37:25.452527+0000","last_peered":"2024-04-26T00:37:25.452527+0000","last_clean":"2024-04-26T00:37:25.452527+0000","last_became_active":"2024-04-26T00:37:25.338693+0000","last_became_peered":"2024-04-26T00:37:25.338693+0000","last_unstale":"2024-04-26T00:37:25.452527+0000","last_undegraded":"2024-04-26T00:37:25.452527+0000","last_fullsized":"2024-04-26T00:37:25.452527+0000","mapping_epoch":59,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:15:47.558025+0000","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3575,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28343,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336632+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:25.336632+0000","last_peered":"2024-04-26T00:37:25.336632+0000","last_clean":"2024-04-26T00:37:25.336632+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:25.336632+0000","last_undegraded":"2024-04-26T00:37:25.336632+0000","last_fullsized":"2024-04-26T00:37:25.336632+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21650,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336183+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:25.336183+0000","last_peered":"2024-04-26T00:37:25.336183+0000","last_clean":"2024-04-26T00:37:25.336183+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:25.336183+0000","last_undegraded":"2024-04-26T00:37:25.336183+0000","last_fullsized":"2024-04-26T00:37:25.336183+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336581+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:25.336581+0000","last_peered":"2024-04-26T00:37:25.336581+0000","last_clean":"2024-04-26T00:37:25.336581+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:25.336581+0000","last_undegraded":"2024-04-26T00:37:25.336581+0000","last_fullsized":"2024-04-26T00:37:25.336581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21673,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:27.484246+0000","last_change":"2024-04-26T00:37:27.484272+0000","last_active":"2024-04-26T00:37:27.484246+0000","last_peered":"2024-04-26T00:37:27.484246+0000","last_clean":"2024-04-26T00:37:27.484246+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:27.484246+0000","last_undegraded":"2024-04-26T00:37:27.484246+0000","last_fullsized":"2024-04-26T00:37:27.484246+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21067","last_scrub_stamp":"2024-04-26T00:37:27.484142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:37:27.484142+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:48:54.707787+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326712+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:24.326712+0000","last_peered":"2024-04-26T00:37:24.326712+0000","last_clean":"2024-04-26T00:37:24.326712+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:24.326712+0000","last_undegraded":"2024-04-26T00:37:24.326712+0000","last_fullsized":"2024-04-26T00:37:24.326712+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28844,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336383+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:25.336383+0000","last_peered":"2024-04-26T00:37:25.336383+0000","last_clean":"2024-04-26T00:37:25.336383+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:25.336383+0000","last_undegraded":"2024-04-26T00:37:25.336383+0000","last_fullsized":"2024-04-26T00:37:25.336383+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":115,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336157+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:25.336157+0000","last_peered":"2024-04-26T00:37:25.336157+0000","last_clean":"2024-04-26T00:37:25.336157+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:25.336157+0000","last_undegraded":"2024-04-26T00:37:25.336157+0000","last_fullsized":"2024-04-26T00:37:25.336157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28920,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326794+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:24.326794+0000","last_peered":"2024-04-26T00:37:24.326794+0000","last_clean":"2024-04-26T00:37:24.326794+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:24.326794+0000","last_undegraded":"2024-04-26T00:37:24.326794+0000","last_fullsized":"2024-04-26T00:37:24.326794+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336630+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:25.336630+0000","last_peered":"2024-04-26T00:37:25.336630+0000","last_clean":"2024-04-26T00:37:25.336630+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:25.336630+0000","last_undegraded":"2024-04-26T00:37:25.336630+0000","last_fullsized":"2024-04-26T00:37:25.336630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28355,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326577+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:24.326577+0000","last_peered":"2024-04-26T00:37:24.326577+0000","last_clean":"2024-04-26T00:37:24.326577+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:24.326577+0000","last_undegraded":"2024-04-26T00:37:24.326577+0000","last_fullsized":"2024-04-26T00:37:24.326577+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22050,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336818+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:25.336818+0000","last_peered":"2024-04-26T00:37:25.336818+0000","last_clean":"2024-04-26T00:37:25.336818+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:25.336818+0000","last_undegraded":"2024-04-26T00:37:25.336818+0000","last_fullsized":"2024-04-26T00:37:25.336818+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337484+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:25.337484+0000","last_peered":"2024-04-26T00:37:25.337484+0000","last_clean":"2024-04-26T00:37:25.337484+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:25.337484+0000","last_undegraded":"2024-04-26T00:37:25.337484+0000","last_fullsized":"2024-04-26T00:37:25.337484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21831,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336989+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:25.336989+0000","last_peered":"2024-04-26T00:37:25.336989+0000","last_clean":"2024-04-26T00:37:25.336989+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:25.336989+0000","last_undegraded":"2024-04-26T00:37:25.336989+0000","last_fullsized":"2024-04-26T00:37:25.336989+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327058+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:24.327058+0000","last_peered":"2024-04-26T00:37:24.327058+0000","last_clean":"2024-04-26T00:37:24.327058+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:24.327058+0000","last_undegraded":"2024-04-26T00:37:24.327058+0000","last_fullsized":"2024-04-26T00:37:24.327058+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22163,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336821+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:25.336821+0000","last_peered":"2024-04-26T00:37:25.336821+0000","last_clean":"2024-04-26T00:37:25.336821+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:25.336821+0000","last_undegraded":"2024-04-26T00:37:25.336821+0000","last_fullsized":"2024-04-26T00:37:25.336821+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336916+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:25.336916+0000","last_peered":"2024-04-26T00:37:25.336916+0000","last_clean":"2024-04-26T00:37:25.336916+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:25.336916+0000","last_undegraded":"2024-04-26T00:37:25.336916+0000","last_fullsized":"2024-04-26T00:37:25.336916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336719+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:25.336719+0000","last_peered":"2024-04-26T00:37:25.336719+0000","last_clean":"2024-04-26T00:37:25.336719+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:25.336719+0000","last_undegraded":"2024-04-26T00:37:25.336719+0000","last_fullsized":"2024-04-26T00:37:25.336719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28771,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337117+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:25.337117+0000","last_peered":"2024-04-26T00:37:25.337117+0000","last_clean":"2024-04-26T00:37:25.337117+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:25.337117+0000","last_undegraded":"2024-04-26T00:37:25.337117+0000","last_fullsized":"2024-04-26T00:37:25.337117+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115261440,"data_stored":115254214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527804,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844036,"statfs":{"total":96636764160,"available":96096292864,"internally_reserved":0,"allocated":503808,"data_stored":168468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":539946703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280670,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546824,"kb_used_data":55676,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93825016,"statfs":{"total":96636764160,"available":96076816384,"internally_reserved":0,"allocated":57012224,"data_stored":56676884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084442,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410244,"kb_used_data":57848,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961596,"statfs":{"total":96636764160,"available":96216674304,"internally_reserved":0,"allocated":59236352,"data_stored":58910108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":56512512,"data_stored":56508899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58744832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:30.202 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-04-26T00:37:30.202 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-26T00:37:30.214 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-26T00:37:30.215 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:30.497 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:30.498 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:30.510 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-04-26T00:37:28.596407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484872,"kb_used_data":114016,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281630648,"statfs":{"total":289910292480,"available":288389783552,"internally_reserved":0,"allocated":116752384,"data_stored":115755460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1403702380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20693,"num_write_kb":-41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2063,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001596"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28641,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337606+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:25.337606+0000","last_peered":"2024-04-26T00:37:25.337606+0000","last_clean":"2024-04-26T00:37:25.337606+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:25.337606+0000","last_undegraded":"2024-04-26T00:37:25.337606+0000","last_fullsized":"2024-04-26T00:37:25.337606+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21810,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327947+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:24.327947+0000","last_peered":"2024-04-26T00:37:24.327947+0000","last_clean":"2024-04-26T00:37:24.327947+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:24.327947+0000","last_undegraded":"2024-04-26T00:37:24.327947+0000","last_fullsized":"2024-04-26T00:37:24.327947+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28595,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337182+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:25.337182+0000","last_peered":"2024-04-26T00:37:25.337182+0000","last_clean":"2024-04-26T00:37:25.337182+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:25.337182+0000","last_undegraded":"2024-04-26T00:37:25.337182+0000","last_fullsized":"2024-04-26T00:37:25.337182+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21182,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326938+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:24.326938+0000","last_peered":"2024-04-26T00:37:24.326938+0000","last_clean":"2024-04-26T00:37:24.326938+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:24.326938+0000","last_undegraded":"2024-04-26T00:37:24.326938+0000","last_fullsized":"2024-04-26T00:37:24.326938+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22030,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337303+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:25.337303+0000","last_peered":"2024-04-26T00:37:25.337303+0000","last_clean":"2024-04-26T00:37:25.337303+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:25.337303+0000","last_undegraded":"2024-04-26T00:37:25.337303+0000","last_fullsized":"2024-04-26T00:37:25.337303+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.452527+0000","last_change":"2024-04-26T00:37:25.452038+0000","last_active":"2024-04-26T00:37:25.452527+0000","last_peered":"2024-04-26T00:37:25.452527+0000","last_clean":"2024-04-26T00:37:25.452527+0000","last_became_active":"2024-04-26T00:37:25.338693+0000","last_became_peered":"2024-04-26T00:37:25.338693+0000","last_unstale":"2024-04-26T00:37:25.452527+0000","last_undegraded":"2024-04-26T00:37:25.452527+0000","last_fullsized":"2024-04-26T00:37:25.452527+0000","mapping_epoch":59,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:15:47.558025+0000","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3575,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28343,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336632+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:25.336632+0000","last_peered":"2024-04-26T00:37:25.336632+0000","last_clean":"2024-04-26T00:37:25.336632+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:25.336632+0000","last_undegraded":"2024-04-26T00:37:25.336632+0000","last_fullsized":"2024-04-26T00:37:25.336632+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21650,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336183+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:25.336183+0000","last_peered":"2024-04-26T00:37:25.336183+0000","last_clean":"2024-04-26T00:37:25.336183+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:25.336183+0000","last_undegraded":"2024-04-26T00:37:25.336183+0000","last_fullsized":"2024-04-26T00:37:25.336183+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336581+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:25.336581+0000","last_peered":"2024-04-26T00:37:25.336581+0000","last_clean":"2024-04-26T00:37:25.336581+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:25.336581+0000","last_undegraded":"2024-04-26T00:37:25.336581+0000","last_fullsized":"2024-04-26T00:37:25.336581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21673,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:27.484246+0000","last_change":"2024-04-26T00:37:27.484272+0000","last_active":"2024-04-26T00:37:27.484246+0000","last_peered":"2024-04-26T00:37:27.484246+0000","last_clean":"2024-04-26T00:37:27.484246+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:27.484246+0000","last_undegraded":"2024-04-26T00:37:27.484246+0000","last_fullsized":"2024-04-26T00:37:27.484246+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21067","last_scrub_stamp":"2024-04-26T00:37:27.484142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:37:27.484142+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:48:54.707787+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326712+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:24.326712+0000","last_peered":"2024-04-26T00:37:24.326712+0000","last_clean":"2024-04-26T00:37:24.326712+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:24.326712+0000","last_undegraded":"2024-04-26T00:37:24.326712+0000","last_fullsized":"2024-04-26T00:37:24.326712+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28844,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336383+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:25.336383+0000","last_peered":"2024-04-26T00:37:25.336383+0000","last_clean":"2024-04-26T00:37:25.336383+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:25.336383+0000","last_undegraded":"2024-04-26T00:37:25.336383+0000","last_fullsized":"2024-04-26T00:37:25.336383+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":115,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336157+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:25.336157+0000","last_peered":"2024-04-26T00:37:25.336157+0000","last_clean":"2024-04-26T00:37:25.336157+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:25.336157+0000","last_undegraded":"2024-04-26T00:37:25.336157+0000","last_fullsized":"2024-04-26T00:37:25.336157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28920,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326794+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:24.326794+0000","last_peered":"2024-04-26T00:37:24.326794+0000","last_clean":"2024-04-26T00:37:24.326794+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:24.326794+0000","last_undegraded":"2024-04-26T00:37:24.326794+0000","last_fullsized":"2024-04-26T00:37:24.326794+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336630+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:25.336630+0000","last_peered":"2024-04-26T00:37:25.336630+0000","last_clean":"2024-04-26T00:37:25.336630+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:25.336630+0000","last_undegraded":"2024-04-26T00:37:25.336630+0000","last_fullsized":"2024-04-26T00:37:25.336630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28355,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.326577+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:24.326577+0000","last_peered":"2024-04-26T00:37:24.326577+0000","last_clean":"2024-04-26T00:37:24.326577+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:24.326577+0000","last_undegraded":"2024-04-26T00:37:24.326577+0000","last_fullsized":"2024-04-26T00:37:24.326577+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22050,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336818+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:25.336818+0000","last_peered":"2024-04-26T00:37:25.336818+0000","last_clean":"2024-04-26T00:37:25.336818+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:25.336818+0000","last_undegraded":"2024-04-26T00:37:25.336818+0000","last_fullsized":"2024-04-26T00:37:25.336818+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337484+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:25.337484+0000","last_peered":"2024-04-26T00:37:25.337484+0000","last_clean":"2024-04-26T00:37:25.337484+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:25.337484+0000","last_undegraded":"2024-04-26T00:37:25.337484+0000","last_fullsized":"2024-04-26T00:37:25.337484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21831,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336989+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:25.336989+0000","last_peered":"2024-04-26T00:37:25.336989+0000","last_clean":"2024-04-26T00:37:25.336989+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:25.336989+0000","last_undegraded":"2024-04-26T00:37:25.336989+0000","last_fullsized":"2024-04-26T00:37:25.336989+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-26T00:37:24.327058+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:24.327058+0000","last_peered":"2024-04-26T00:37:24.327058+0000","last_clean":"2024-04-26T00:37:24.327058+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:24.327058+0000","last_undegraded":"2024-04-26T00:37:24.327058+0000","last_fullsized":"2024-04-26T00:37:24.327058+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22163,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336821+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:25.336821+0000","last_peered":"2024-04-26T00:37:25.336821+0000","last_clean":"2024-04-26T00:37:25.336821+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:25.336821+0000","last_undegraded":"2024-04-26T00:37:25.336821+0000","last_fullsized":"2024-04-26T00:37:25.336821+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336916+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:25.336916+0000","last_peered":"2024-04-26T00:37:25.336916+0000","last_clean":"2024-04-26T00:37:25.336916+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:25.336916+0000","last_undegraded":"2024-04-26T00:37:25.336916+0000","last_fullsized":"2024-04-26T00:37:25.336916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336719+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:25.336719+0000","last_peered":"2024-04-26T00:37:25.336719+0000","last_clean":"2024-04-26T00:37:25.336719+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:25.336719+0000","last_undegraded":"2024-04-26T00:37:25.336719+0000","last_fullsized":"2024-04-26T00:37:25.336719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28771,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337117+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:25.337117+0000","last_peered":"2024-04-26T00:37:25.337117+0000","last_clean":"2024-04-26T00:37:25.337117+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:25.337117+0000","last_undegraded":"2024-04-26T00:37:25.337117+0000","last_fullsized":"2024-04-26T00:37:25.337117+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115261440,"data_stored":115254214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527804,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844036,"statfs":{"total":96636764160,"available":96096292864,"internally_reserved":0,"allocated":503808,"data_stored":168468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":539946703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280670,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546824,"kb_used_data":55676,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93825016,"statfs":{"total":96636764160,"available":96076816384,"internally_reserved":0,"allocated":57012224,"data_stored":56676884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084442,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410244,"kb_used_data":57848,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93961596,"statfs":{"total":96636764160,"available":96216674304,"internally_reserved":0,"allocated":59236352,"data_stored":58910108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":56512512,"data_stored":56508899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":58744832,"data_stored":58744832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:30.511 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:30.786 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:30.786 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:30.798 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-04-26T00:37:30.596738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480080,"kb_used_data":109228,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281635440,"statfs":{"total":289910292480,"available":288394690560,"internally_reserved":0,"allocated":111849472,"data_stored":110847065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1403702380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20693,"num_write_kb":-41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3060,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001595"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28641,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337606+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:25.337606+0000","last_peered":"2024-04-26T00:37:25.337606+0000","last_clean":"2024-04-26T00:37:25.337606+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:25.337606+0000","last_undegraded":"2024-04-26T00:37:25.337606+0000","last_fullsized":"2024-04-26T00:37:25.337606+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21812,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333599+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:25.333599+0000","last_peered":"2024-04-26T00:37:25.333599+0000","last_clean":"2024-04-26T00:37:25.333599+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:25.333599+0000","last_undegraded":"2024-04-26T00:37:25.333599+0000","last_fullsized":"2024-04-26T00:37:25.333599+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28595,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337182+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:25.337182+0000","last_peered":"2024-04-26T00:37:25.337182+0000","last_clean":"2024-04-26T00:37:25.337182+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:25.337182+0000","last_undegraded":"2024-04-26T00:37:25.337182+0000","last_fullsized":"2024-04-26T00:37:25.337182+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21184,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333645+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:25.333645+0000","last_peered":"2024-04-26T00:37:25.333645+0000","last_clean":"2024-04-26T00:37:25.333645+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:25.333645+0000","last_undegraded":"2024-04-26T00:37:25.333645+0000","last_fullsized":"2024-04-26T00:37:25.333645+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22030,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337303+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:25.337303+0000","last_peered":"2024-04-26T00:37:25.337303+0000","last_clean":"2024-04-26T00:37:25.337303+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:25.337303+0000","last_undegraded":"2024-04-26T00:37:25.337303+0000","last_fullsized":"2024-04-26T00:37:25.337303+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.452527+0000","last_change":"2024-04-26T00:37:25.452038+0000","last_active":"2024-04-26T00:37:25.452527+0000","last_peered":"2024-04-26T00:37:25.452527+0000","last_clean":"2024-04-26T00:37:25.452527+0000","last_became_active":"2024-04-26T00:37:25.338693+0000","last_became_peered":"2024-04-26T00:37:25.338693+0000","last_unstale":"2024-04-26T00:37:25.452527+0000","last_undegraded":"2024-04-26T00:37:25.452527+0000","last_fullsized":"2024-04-26T00:37:25.452527+0000","mapping_epoch":59,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:15:47.558025+0000","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3575,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28343,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336632+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:25.336632+0000","last_peered":"2024-04-26T00:37:25.336632+0000","last_clean":"2024-04-26T00:37:25.336632+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:25.336632+0000","last_undegraded":"2024-04-26T00:37:25.336632+0000","last_fullsized":"2024-04-26T00:37:25.336632+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21650,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336183+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:25.336183+0000","last_peered":"2024-04-26T00:37:25.336183+0000","last_clean":"2024-04-26T00:37:25.336183+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:25.336183+0000","last_undegraded":"2024-04-26T00:37:25.336183+0000","last_fullsized":"2024-04-26T00:37:25.336183+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336581+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:25.336581+0000","last_peered":"2024-04-26T00:37:25.336581+0000","last_clean":"2024-04-26T00:37:25.336581+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:25.336581+0000","last_undegraded":"2024-04-26T00:37:25.336581+0000","last_fullsized":"2024-04-26T00:37:25.336581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21673,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:27.484246+0000","last_change":"2024-04-26T00:37:27.484272+0000","last_active":"2024-04-26T00:37:27.484246+0000","last_peered":"2024-04-26T00:37:27.484246+0000","last_clean":"2024-04-26T00:37:27.484246+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:27.484246+0000","last_undegraded":"2024-04-26T00:37:27.484246+0000","last_fullsized":"2024-04-26T00:37:27.484246+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21067","last_scrub_stamp":"2024-04-26T00:37:27.484142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:37:27.484142+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:48:54.707787+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334182+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:25.334182+0000","last_peered":"2024-04-26T00:37:25.334182+0000","last_clean":"2024-04-26T00:37:25.334182+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:25.334182+0000","last_undegraded":"2024-04-26T00:37:25.334182+0000","last_fullsized":"2024-04-26T00:37:25.334182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28844,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336383+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:25.336383+0000","last_peered":"2024-04-26T00:37:25.336383+0000","last_clean":"2024-04-26T00:37:25.336383+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:25.336383+0000","last_undegraded":"2024-04-26T00:37:25.336383+0000","last_fullsized":"2024-04-26T00:37:25.336383+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":115,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336157+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:25.336157+0000","last_peered":"2024-04-26T00:37:25.336157+0000","last_clean":"2024-04-26T00:37:25.336157+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:25.336157+0000","last_undegraded":"2024-04-26T00:37:25.336157+0000","last_fullsized":"2024-04-26T00:37:25.336157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28922,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334284+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:25.334284+0000","last_peered":"2024-04-26T00:37:25.334284+0000","last_clean":"2024-04-26T00:37:25.334284+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:25.334284+0000","last_undegraded":"2024-04-26T00:37:25.334284+0000","last_fullsized":"2024-04-26T00:37:25.334284+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336630+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:25.336630+0000","last_peered":"2024-04-26T00:37:25.336630+0000","last_clean":"2024-04-26T00:37:25.336630+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:25.336630+0000","last_undegraded":"2024-04-26T00:37:25.336630+0000","last_fullsized":"2024-04-26T00:37:25.336630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28357,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333409+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:25.333409+0000","last_peered":"2024-04-26T00:37:25.333409+0000","last_clean":"2024-04-26T00:37:25.333409+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:25.333409+0000","last_undegraded":"2024-04-26T00:37:25.333409+0000","last_fullsized":"2024-04-26T00:37:25.333409+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22050,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336818+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:25.336818+0000","last_peered":"2024-04-26T00:37:25.336818+0000","last_clean":"2024-04-26T00:37:25.336818+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:25.336818+0000","last_undegraded":"2024-04-26T00:37:25.336818+0000","last_fullsized":"2024-04-26T00:37:25.336818+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337484+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:25.337484+0000","last_peered":"2024-04-26T00:37:25.337484+0000","last_clean":"2024-04-26T00:37:25.337484+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:25.337484+0000","last_undegraded":"2024-04-26T00:37:25.337484+0000","last_fullsized":"2024-04-26T00:37:25.337484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21831,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336989+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:25.336989+0000","last_peered":"2024-04-26T00:37:25.336989+0000","last_clean":"2024-04-26T00:37:25.336989+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:25.336989+0000","last_undegraded":"2024-04-26T00:37:25.336989+0000","last_fullsized":"2024-04-26T00:37:25.336989+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334933+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:25.334933+0000","last_peered":"2024-04-26T00:37:25.334933+0000","last_clean":"2024-04-26T00:37:25.334933+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:25.334933+0000","last_undegraded":"2024-04-26T00:37:25.334933+0000","last_fullsized":"2024-04-26T00:37:25.334933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22163,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336821+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:25.336821+0000","last_peered":"2024-04-26T00:37:25.336821+0000","last_clean":"2024-04-26T00:37:25.336821+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:25.336821+0000","last_undegraded":"2024-04-26T00:37:25.336821+0000","last_fullsized":"2024-04-26T00:37:25.336821+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336916+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:25.336916+0000","last_peered":"2024-04-26T00:37:25.336916+0000","last_clean":"2024-04-26T00:37:25.336916+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:25.336916+0000","last_undegraded":"2024-04-26T00:37:25.336916+0000","last_fullsized":"2024-04-26T00:37:25.336916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336719+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:25.336719+0000","last_peered":"2024-04-26T00:37:25.336719+0000","last_clean":"2024-04-26T00:37:25.336719+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:25.336719+0000","last_undegraded":"2024-04-26T00:37:25.336719+0000","last_fullsized":"2024-04-26T00:37:25.336719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28771,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337117+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:25.337117+0000","last_peered":"2024-04-26T00:37:25.337117+0000","last_clean":"2024-04-26T00:37:25.337117+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:25.337117+0000","last_undegraded":"2024-04-26T00:37:25.337117+0000","last_fullsized":"2024-04-26T00:37:25.337117+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110350336,"data_stored":110343110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527804,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844036,"statfs":{"total":96636764160,"available":96096292864,"internally_reserved":0,"allocated":503808,"data_stored":168468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":539946703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280670,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546824,"kb_used_data":55676,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93825016,"statfs":{"total":96636764160,"available":96076816384,"internally_reserved":0,"allocated":57012224,"data_stored":56676884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084443,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405452,"kb_used_data":53060,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93966388,"statfs":{"total":96636764160,"available":96221581312,"internally_reserved":0,"allocated":54333440,"data_stored":54001713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":56512512,"data_stored":56508899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53833728,"data_stored":53833728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:30.799 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-26T00:37:30.799 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:31.074 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:31.075 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:31.086 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-04-26T00:37:30.596738+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480080,"kb_used_data":109228,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281635440,"statfs":{"total":289910292480,"available":288394690560,"internally_reserved":0,"allocated":111849472,"data_stored":110847065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1403702380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20693,"num_write_kb":-41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3060,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001595"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28641,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337606+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:25.337606+0000","last_peered":"2024-04-26T00:37:25.337606+0000","last_clean":"2024-04-26T00:37:25.337606+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:25.337606+0000","last_undegraded":"2024-04-26T00:37:25.337606+0000","last_fullsized":"2024-04-26T00:37:25.337606+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21812,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333599+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:25.333599+0000","last_peered":"2024-04-26T00:37:25.333599+0000","last_clean":"2024-04-26T00:37:25.333599+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:25.333599+0000","last_undegraded":"2024-04-26T00:37:25.333599+0000","last_fullsized":"2024-04-26T00:37:25.333599+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28595,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337182+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:25.337182+0000","last_peered":"2024-04-26T00:37:25.337182+0000","last_clean":"2024-04-26T00:37:25.337182+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:25.337182+0000","last_undegraded":"2024-04-26T00:37:25.337182+0000","last_fullsized":"2024-04-26T00:37:25.337182+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21184,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333645+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:25.333645+0000","last_peered":"2024-04-26T00:37:25.333645+0000","last_clean":"2024-04-26T00:37:25.333645+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:25.333645+0000","last_undegraded":"2024-04-26T00:37:25.333645+0000","last_fullsized":"2024-04-26T00:37:25.333645+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22030,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337303+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:25.337303+0000","last_peered":"2024-04-26T00:37:25.337303+0000","last_clean":"2024-04-26T00:37:25.337303+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:25.337303+0000","last_undegraded":"2024-04-26T00:37:25.337303+0000","last_fullsized":"2024-04-26T00:37:25.337303+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.452527+0000","last_change":"2024-04-26T00:37:25.452038+0000","last_active":"2024-04-26T00:37:25.452527+0000","last_peered":"2024-04-26T00:37:25.452527+0000","last_clean":"2024-04-26T00:37:25.452527+0000","last_became_active":"2024-04-26T00:37:25.338693+0000","last_became_peered":"2024-04-26T00:37:25.338693+0000","last_unstale":"2024-04-26T00:37:25.452527+0000","last_undegraded":"2024-04-26T00:37:25.452527+0000","last_fullsized":"2024-04-26T00:37:25.452527+0000","mapping_epoch":59,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:15:47.558025+0000","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3575,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28343,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336632+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:25.336632+0000","last_peered":"2024-04-26T00:37:25.336632+0000","last_clean":"2024-04-26T00:37:25.336632+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:25.336632+0000","last_undegraded":"2024-04-26T00:37:25.336632+0000","last_fullsized":"2024-04-26T00:37:25.336632+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21650,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336183+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:25.336183+0000","last_peered":"2024-04-26T00:37:25.336183+0000","last_clean":"2024-04-26T00:37:25.336183+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:25.336183+0000","last_undegraded":"2024-04-26T00:37:25.336183+0000","last_fullsized":"2024-04-26T00:37:25.336183+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336581+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:25.336581+0000","last_peered":"2024-04-26T00:37:25.336581+0000","last_clean":"2024-04-26T00:37:25.336581+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:25.336581+0000","last_undegraded":"2024-04-26T00:37:25.336581+0000","last_fullsized":"2024-04-26T00:37:25.336581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21673,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:27.484246+0000","last_change":"2024-04-26T00:37:27.484272+0000","last_active":"2024-04-26T00:37:27.484246+0000","last_peered":"2024-04-26T00:37:27.484246+0000","last_clean":"2024-04-26T00:37:27.484246+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:27.484246+0000","last_undegraded":"2024-04-26T00:37:27.484246+0000","last_fullsized":"2024-04-26T00:37:27.484246+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21067","last_scrub_stamp":"2024-04-26T00:37:27.484142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:37:27.484142+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:48:54.707787+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334182+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:25.334182+0000","last_peered":"2024-04-26T00:37:25.334182+0000","last_clean":"2024-04-26T00:37:25.334182+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:25.334182+0000","last_undegraded":"2024-04-26T00:37:25.334182+0000","last_fullsized":"2024-04-26T00:37:25.334182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28844,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336383+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:25.336383+0000","last_peered":"2024-04-26T00:37:25.336383+0000","last_clean":"2024-04-26T00:37:25.336383+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:25.336383+0000","last_undegraded":"2024-04-26T00:37:25.336383+0000","last_fullsized":"2024-04-26T00:37:25.336383+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":115,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336157+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:25.336157+0000","last_peered":"2024-04-26T00:37:25.336157+0000","last_clean":"2024-04-26T00:37:25.336157+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:25.336157+0000","last_undegraded":"2024-04-26T00:37:25.336157+0000","last_fullsized":"2024-04-26T00:37:25.336157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28922,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334284+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:25.334284+0000","last_peered":"2024-04-26T00:37:25.334284+0000","last_clean":"2024-04-26T00:37:25.334284+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:25.334284+0000","last_undegraded":"2024-04-26T00:37:25.334284+0000","last_fullsized":"2024-04-26T00:37:25.334284+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336630+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:25.336630+0000","last_peered":"2024-04-26T00:37:25.336630+0000","last_clean":"2024-04-26T00:37:25.336630+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:25.336630+0000","last_undegraded":"2024-04-26T00:37:25.336630+0000","last_fullsized":"2024-04-26T00:37:25.336630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28357,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333409+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:25.333409+0000","last_peered":"2024-04-26T00:37:25.333409+0000","last_clean":"2024-04-26T00:37:25.333409+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:25.333409+0000","last_undegraded":"2024-04-26T00:37:25.333409+0000","last_fullsized":"2024-04-26T00:37:25.333409+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22050,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336818+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:25.336818+0000","last_peered":"2024-04-26T00:37:25.336818+0000","last_clean":"2024-04-26T00:37:25.336818+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:25.336818+0000","last_undegraded":"2024-04-26T00:37:25.336818+0000","last_fullsized":"2024-04-26T00:37:25.336818+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337484+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:25.337484+0000","last_peered":"2024-04-26T00:37:25.337484+0000","last_clean":"2024-04-26T00:37:25.337484+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:25.337484+0000","last_undegraded":"2024-04-26T00:37:25.337484+0000","last_fullsized":"2024-04-26T00:37:25.337484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21831,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336989+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:25.336989+0000","last_peered":"2024-04-26T00:37:25.336989+0000","last_clean":"2024-04-26T00:37:25.336989+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:25.336989+0000","last_undegraded":"2024-04-26T00:37:25.336989+0000","last_fullsized":"2024-04-26T00:37:25.336989+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334933+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:25.334933+0000","last_peered":"2024-04-26T00:37:25.334933+0000","last_clean":"2024-04-26T00:37:25.334933+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:25.334933+0000","last_undegraded":"2024-04-26T00:37:25.334933+0000","last_fullsized":"2024-04-26T00:37:25.334933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22163,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336821+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:25.336821+0000","last_peered":"2024-04-26T00:37:25.336821+0000","last_clean":"2024-04-26T00:37:25.336821+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:25.336821+0000","last_undegraded":"2024-04-26T00:37:25.336821+0000","last_fullsized":"2024-04-26T00:37:25.336821+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336916+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:25.336916+0000","last_peered":"2024-04-26T00:37:25.336916+0000","last_clean":"2024-04-26T00:37:25.336916+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:25.336916+0000","last_undegraded":"2024-04-26T00:37:25.336916+0000","last_fullsized":"2024-04-26T00:37:25.336916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336719+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:25.336719+0000","last_peered":"2024-04-26T00:37:25.336719+0000","last_clean":"2024-04-26T00:37:25.336719+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:25.336719+0000","last_undegraded":"2024-04-26T00:37:25.336719+0000","last_fullsized":"2024-04-26T00:37:25.336719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28771,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337117+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:25.337117+0000","last_peered":"2024-04-26T00:37:25.337117+0000","last_clean":"2024-04-26T00:37:25.337117+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:25.337117+0000","last_undegraded":"2024-04-26T00:37:25.337117+0000","last_fullsized":"2024-04-26T00:37:25.337117+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110350336,"data_stored":110343110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527804,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844036,"statfs":{"total":96636764160,"available":96096292864,"internally_reserved":0,"allocated":503808,"data_stored":168468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":539946703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280670,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546824,"kb_used_data":55676,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93825016,"statfs":{"total":96636764160,"available":96076816384,"internally_reserved":0,"allocated":57012224,"data_stored":56676884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084443,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405452,"kb_used_data":53060,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93966388,"statfs":{"total":96636764160,"available":96221581312,"internally_reserved":0,"allocated":54333440,"data_stored":54001713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":56512512,"data_stored":56508899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53833728,"data_stored":53833728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:31.088 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T00:37:31.376 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:31.376 INFO:teuthology.orchestra.run.smithi072.stdout:{"epoch":60,"fsid":"fecf292c-395a-497e-b8b4-a892b12f3dd9","created":"2024-04-26T00:24:14.092045+0000","modified":"2024-04-26T00:37:25.322978+0000","last_up_change":"2024-04-26T00:25:59.320810+0000","last_in_change":"2024-04-26T00:25:53.266636+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-26T00:24:22.712820+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-04-26T00:24:31.936865+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"264b339b-fab7-4705-92f1-2b72519db1a7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":18,"up_from":22,"up_thru":58,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6800","nonce":2078650556}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6801","nonce":2078650556}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6803","nonce":2078650556}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6802","nonce":2078650556}]},"public_addr":"172.21.15.72:6800/2078650556","cluster_addr":"172.21.15.72:6801/2078650556","heartbeat_back_addr":"172.21.15.72:6803/2078650556","heartbeat_front_addr":"172.21.15.72:6802/2078650556","state":["exists","up"]},{"osd":1,"uuid":"200d0549-873b-4a17-b213-8c659dd090d5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":23,"up_from":34,"up_thru":59,"down_at":24,"lost_at":24,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6808","nonce":2570455510}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6809","nonce":2570455510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6811","nonce":2570455510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6810","nonce":2570455510}]},"public_addr":"172.21.15.72:6808/2570455510","cluster_addr":"172.21.15.72:6809/2570455510","heartbeat_back_addr":"172.21.15.72:6811/2570455510","heartbeat_front_addr":"172.21.15.72:6810/2570455510","state":["exists","up"]},{"osd":2,"uuid":"f0d45389-e221-4b90-90d6-79aeafc0effd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":14,"up_from":28,"up_thru":48,"down_at":16,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6804","nonce":3995376740}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6805","nonce":3995376740}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6807","nonce":3995376740}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.72:6806","nonce":3995376740}]},"public_addr":"172.21.15.72:6804/3995376740","cluster_addr":"172.21.15.72:6805/3995376740","heartbeat_back_addr":"172.21.15.72:6807/3995376740","heartbeat_front_addr":"172.21.15.72:6806/3995376740","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.217902+0000","dead_epoch":19},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.289667+0000","dead_epoch":24},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T00:24:20.265432+0000","dead_epoch":16}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T00:37:32.390 INFO:tasks.ceph:Scrubbing osd.0 2024-04-26T00:37:32.390 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-26T00:37:32.520 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:37:32.520 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T00:37:32.520 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:37:32.531 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-26T00:37:32.811 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 0 to deep-scrub 2024-04-26T00:37:32.824 INFO:tasks.ceph:Scrubbing osd.1 2024-04-26T00:37:32.824 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-26T00:37:32.942 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:37:32.943 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T00:37:32.943 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:37:32.953 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-26T00:37:33.238 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 1 to deep-scrub 2024-04-26T00:37:33.251 INFO:tasks.ceph:Scrubbing osd.2 2024-04-26T00:37:33.251 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-26T00:37:33.372 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-04-26T00:37:33.372 INFO:teuthology.orchestra.run.smithi072.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-26T00:37:33.372 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-04-26T00:37:33.383 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-26T00:37:33.661 INFO:teuthology.orchestra.run.smithi072.stderr:instructed osd(s) 2 to deep-scrub 2024-04-26T00:37:33.674 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:33.951 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:33.951 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:33.963 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":453,"stamp":"2024-04-26T00:37:32.597027+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477676,"kb_used_data":106828,"kb_used_omap":12,"kb_used_meta":1370803,"kb_avail":281637844,"statfs":{"total":289910292480,"available":288397152256,"internally_reserved":0,"allocated":109391872,"data_stored":108389465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1403702380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-966,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-407,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-20693,"num_write_kb":-41380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-3060,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.271249"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28641,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337606+0000","last_change":"2024-04-26T00:35:29.015703+0000","last_active":"2024-04-26T00:37:25.337606+0000","last_peered":"2024-04-26T00:37:25.337606+0000","last_clean":"2024-04-26T00:37:25.337606+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:25.337606+0000","last_undegraded":"2024-04-26T00:37:25.337606+0000","last_fullsized":"2024-04-26T00:37:25.337606+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:22:03.368134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21812,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333599+0000","last_change":"2024-04-26T00:25:44.635538+0000","last_active":"2024-04-26T00:37:25.333599+0000","last_peered":"2024-04-26T00:37:25.333599+0000","last_clean":"2024-04-26T00:37:25.333599+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:25.333599+0000","last_undegraded":"2024-04-26T00:37:25.333599+0000","last_fullsized":"2024-04-26T00:37:25.333599+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2930","last_scrub_stamp":"2024-04-26T00:25:44.635370+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:44.635370+0000","objects_scrubbed":2889,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:06:04.218928+0000","scrub_duration":684,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28595,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337182+0000","last_change":"2024-04-26T00:31:52.389971+0000","last_active":"2024-04-26T00:37:25.337182+0000","last_peered":"2024-04-26T00:37:25.337182+0000","last_clean":"2024-04-26T00:37:25.337182+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:25.337182+0000","last_undegraded":"2024-04-26T00:37:25.337182+0000","last_fullsized":"2024-04-26T00:37:25.337182+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2843","last_scrub_stamp":"2024-04-26T00:25:45.686694+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.686694+0000","objects_scrubbed":2808,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:46:15.512082+0000","scrub_duration":770,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21184,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333645+0000","last_change":"2024-04-26T00:25:08.969528+0000","last_active":"2024-04-26T00:37:25.333645+0000","last_peered":"2024-04-26T00:37:25.333645+0000","last_clean":"2024-04-26T00:37:25.333645+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:25.333645+0000","last_undegraded":"2024-04-26T00:37:25.333645+0000","last_fullsized":"2024-04-26T00:37:25.333645+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:24:32.507794+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:29:50.696599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22030,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337303+0000","last_change":"2024-04-26T00:25:54.384693+0000","last_active":"2024-04-26T00:37:25.337303+0000","last_peered":"2024-04-26T00:37:25.337303+0000","last_clean":"2024-04-26T00:37:25.337303+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:25.337303+0000","last_undegraded":"2024-04-26T00:37:25.337303+0000","last_fullsized":"2024-04-26T00:37:25.337303+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'3704","last_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_deep_scrub":"31'3704","last_deep_scrub_stamp":"2024-04-26T00:25:54.384548+0000","last_clean_scrub_stamp":"2024-04-26T00:25:54.384548+0000","objects_scrubbed":3271,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":7,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:37:45.731285+0000","scrub_duration":6457,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22074,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.452527+0000","last_change":"2024-04-26T00:37:25.452038+0000","last_active":"2024-04-26T00:37:25.452527+0000","last_peered":"2024-04-26T00:37:25.452527+0000","last_clean":"2024-04-26T00:37:25.452527+0000","last_became_active":"2024-04-26T00:37:25.338693+0000","last_became_peered":"2024-04-26T00:37:25.338693+0000","last_unstale":"2024-04-26T00:37:25.452527+0000","last_undegraded":"2024-04-26T00:37:25.452527+0000","last_fullsized":"2024-04-26T00:37:25.452527+0000","mapping_epoch":59,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2845","last_scrub_stamp":"2024-04-26T00:25:46.819941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:46.819941+0000","objects_scrubbed":2820,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:15:47.558025+0000","scrub_duration":941,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3575,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28343,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336632+0000","last_change":"2024-04-26T00:29:56.901952+0000","last_active":"2024-04-26T00:37:25.336632+0000","last_peered":"2024-04-26T00:37:25.336632+0000","last_clean":"2024-04-26T00:37:25.336632+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:25.336632+0000","last_undegraded":"2024-04-26T00:37:25.336632+0000","last_fullsized":"2024-04-26T00:37:25.336632+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2827","last_scrub_stamp":"2024-04-26T00:25:45.706348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:45.706348+0000","objects_scrubbed":2792,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:03:51.604902+0000","scrub_duration":825,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21650,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336183+0000","last_change":"2024-04-26T00:25:48.898394+0000","last_active":"2024-04-26T00:37:25.336183+0000","last_peered":"2024-04-26T00:37:25.336183+0000","last_clean":"2024-04-26T00:37:25.336183+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:25.336183+0000","last_undegraded":"2024-04-26T00:37:25.336183+0000","last_fullsized":"2024-04-26T00:37:25.336183+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2969","last_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_deep_scrub":"29'2969","last_deep_scrub_stamp":"2024-04-26T00:25:48.898277+0000","last_clean_scrub_stamp":"2024-04-26T00:25:48.898277+0000","objects_scrubbed":2800,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":6,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:38:40.502600+0000","scrub_duration":5048,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336581+0000","last_change":"2024-04-26T00:26:01.364312+0000","last_active":"2024-04-26T00:37:25.336581+0000","last_peered":"2024-04-26T00:37:25.336581+0000","last_clean":"2024-04-26T00:37:25.336581+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:25.336581+0000","last_undegraded":"2024-04-26T00:37:25.336581+0000","last_fullsized":"2024-04-26T00:37:25.336581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:59:00.048995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21673,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:27.484246+0000","last_change":"2024-04-26T00:37:27.484272+0000","last_active":"2024-04-26T00:37:27.484246+0000","last_peered":"2024-04-26T00:37:27.484246+0000","last_clean":"2024-04-26T00:37:27.484246+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:27.484246+0000","last_undegraded":"2024-04-26T00:37:27.484246+0000","last_fullsized":"2024-04-26T00:37:27.484246+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21067","last_scrub_stamp":"2024-04-26T00:37:27.484142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:37:27.484142+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:48:54.707787+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334182+0000","last_change":"2024-04-26T00:25:08.969847+0000","last_active":"2024-04-26T00:37:25.334182+0000","last_peered":"2024-04-26T00:37:25.334182+0000","last_clean":"2024-04-26T00:37:25.334182+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:25.334182+0000","last_undegraded":"2024-04-26T00:37:25.334182+0000","last_fullsized":"2024-04-26T00:37:25.334182+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:40:37.190748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28844,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336383+0000","last_change":"2024-04-26T00:36:08.134769+0000","last_active":"2024-04-26T00:37:25.336383+0000","last_peered":"2024-04-26T00:37:25.336383+0000","last_clean":"2024-04-26T00:37:25.336383+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:25.336383+0000","last_undegraded":"2024-04-26T00:37:25.336383+0000","last_fullsized":"2024-04-26T00:37:25.336383+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3241","last_scrub_stamp":"2024-04-26T00:25:51.982004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:51.982004+0000","objects_scrubbed":3211,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:58:20.215679+0000","scrub_duration":965,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":115,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336157+0000","last_change":"2024-04-26T00:26:01.455158+0000","last_active":"2024-04-26T00:37:25.336157+0000","last_peered":"2024-04-26T00:37:25.336157+0000","last_clean":"2024-04-26T00:37:25.336157+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:25.336157+0000","last_undegraded":"2024-04-26T00:37:25.336157+0000","last_fullsized":"2024-04-26T00:37:25.336157+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:12:05.667031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28922,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334284+0000","last_change":"2024-04-26T00:35:09.008840+0000","last_active":"2024-04-26T00:37:25.334284+0000","last_peered":"2024-04-26T00:37:25.334284+0000","last_clean":"2024-04-26T00:37:25.334284+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:25.334284+0000","last_undegraded":"2024-04-26T00:37:25.334284+0000","last_fullsized":"2024-04-26T00:37:25.334284+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3022","last_scrub_stamp":"2024-04-26T00:25:47.736821+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.736821+0000","objects_scrubbed":2995,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:31:35.453151+0000","scrub_duration":758,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336630+0000","last_change":"2024-04-26T00:26:01.433765+0000","last_active":"2024-04-26T00:37:25.336630+0000","last_peered":"2024-04-26T00:37:25.336630+0000","last_clean":"2024-04-26T00:37:25.336630+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:25.336630+0000","last_undegraded":"2024-04-26T00:37:25.336630+0000","last_fullsized":"2024-04-26T00:37:25.336630+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:42:05.169929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28357,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.333409+0000","last_change":"2024-04-26T00:27:38.092738+0000","last_active":"2024-04-26T00:37:25.333409+0000","last_peered":"2024-04-26T00:37:25.333409+0000","last_clean":"2024-04-26T00:37:25.333409+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:25.333409+0000","last_undegraded":"2024-04-26T00:37:25.333409+0000","last_fullsized":"2024-04-26T00:37:25.333409+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3043","last_scrub_stamp":"2024-04-26T00:25:49.302264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.302264+0000","objects_scrubbed":2989,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:37:24.370522+0000","scrub_duration":1317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22050,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336818+0000","last_change":"2024-04-26T00:37:05.333868+0000","last_active":"2024-04-26T00:37:25.336818+0000","last_peered":"2024-04-26T00:37:25.336818+0000","last_clean":"2024-04-26T00:37:25.336818+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:25.336818+0000","last_undegraded":"2024-04-26T00:37:25.336818+0000","last_fullsized":"2024-04-26T00:37:25.336818+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3099","last_scrub_stamp":"2024-04-26T00:25:49.772261+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:49.772261+0000","objects_scrubbed":3077,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:50:29.341271+0000","scrub_duration":803,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337484+0000","last_change":"2024-04-26T00:26:01.486290+0000","last_active":"2024-04-26T00:37:25.337484+0000","last_peered":"2024-04-26T00:37:25.337484+0000","last_clean":"2024-04-26T00:37:25.337484+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:25.337484+0000","last_undegraded":"2024-04-26T00:37:25.337484+0000","last_fullsized":"2024-04-26T00:37:25.337484+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:00:10.344723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21831,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336989+0000","last_change":"2024-04-26T00:36:46.280671+0000","last_active":"2024-04-26T00:37:25.336989+0000","last_peered":"2024-04-26T00:37:25.336989+0000","last_clean":"2024-04-26T00:37:25.336989+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:25.336989+0000","last_undegraded":"2024-04-26T00:37:25.336989+0000","last_fullsized":"2024-04-26T00:37:25.336989+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3145","last_scrub_stamp":"2024-04-26T00:25:50.799180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:50.799180+0000","objects_scrubbed":3110,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:22:54.346295+0000","scrub_duration":840,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.334933+0000","last_change":"2024-04-26T00:25:08.975493+0000","last_active":"2024-04-26T00:37:25.334933+0000","last_peered":"2024-04-26T00:37:25.334933+0000","last_clean":"2024-04-26T00:37:25.334933+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:25.334933+0000","last_undegraded":"2024-04-26T00:37:25.334933+0000","last_fullsized":"2024-04-26T00:37:25.334933+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T11:20:56.359722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22163,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336821+0000","last_change":"2024-04-26T00:36:26.240650+0000","last_active":"2024-04-26T00:37:25.336821+0000","last_peered":"2024-04-26T00:37:25.336821+0000","last_clean":"2024-04-26T00:37:25.336821+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:25.336821+0000","last_undegraded":"2024-04-26T00:37:25.336821+0000","last_fullsized":"2024-04-26T00:37:25.336821+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'3325","last_scrub_stamp":"2024-04-26T00:25:53.106632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:53.106632+0000","objects_scrubbed":3300,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:12:46.686858+0000","scrub_duration":1077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":104,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336916+0000","last_change":"2024-04-26T00:26:01.486585+0000","last_active":"2024-04-26T00:37:25.336916+0000","last_peered":"2024-04-26T00:37:25.336916+0000","last_clean":"2024-04-26T00:37:25.336916+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:25.336916+0000","last_undegraded":"2024-04-26T00:37:25.336916+0000","last_fullsized":"2024-04-26T00:37:25.336916+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:21:57.166932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":108,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.336719+0000","last_change":"2024-04-26T00:26:01.370658+0000","last_active":"2024-04-26T00:37:25.336719+0000","last_peered":"2024-04-26T00:37:25.336719+0000","last_clean":"2024-04-26T00:37:25.336719+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:25.336719+0000","last_undegraded":"2024-04-26T00:37:25.336719+0000","last_fullsized":"2024-04-26T00:37:25.336719+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:23.468323+0000","last_clean_scrub_stamp":"2024-04-26T00:24:23.468323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:29:41.500900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"44'21322","reported_seq":28771,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:25.337117+0000","last_change":"2024-04-26T00:33:56.495870+0000","last_active":"2024-04-26T00:37:25.337117+0000","last_peered":"2024-04-26T00:37:25.337117+0000","last_clean":"2024-04-26T00:37:25.337117+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:25.337117+0000","last_undegraded":"2024-04-26T00:37:25.337117+0000","last_fullsized":"2024-04-26T00:37:25.337117+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'2975","last_scrub_stamp":"2024-04-26T00:25:47.549742+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:24:32.507794+0000","last_clean_scrub_stamp":"2024-04-26T00:25:47.549742+0000","objects_scrubbed":2939,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:33:03.039215+0000","scrub_duration":674,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":107892736,"data_stored":107885510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527804,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844036,"statfs":{"total":96636764160,"available":96096292864,"internally_reserved":0,"allocated":503808,"data_stored":168468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":539946703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280671,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544420,"kb_used_data":53276,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93827420,"statfs":{"total":96636764160,"available":96079278080,"internally_reserved":0,"allocated":54554624,"data_stored":54219284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":502918861},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084443,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":405452,"kb_used_data":53060,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93966388,"statfs":{"total":96636764160,"available":96221581312,"internally_reserved":0,"allocated":54333440,"data_stored":54001713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":54054912,"data_stored":54051299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":53833728,"data_stored":53833728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:33.967 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-04-26T00:24:32.507794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.967 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-04-26T00:25:44.635370+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=44, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.967 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-04-26T00:25:45.686694+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=45, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-04-26T00:24:32.507794+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=32, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-04-26T00:25:54.384548+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=54, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-04-26T00:25:46.819941+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=46, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-04-26T00:25:45.706348+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=45, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-04-26T00:25:48.898277+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=48, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-26T00:24:23.468323+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=23, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-26T00:37:27.484142+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=27, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-26T00:24:23.468323+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=23, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.968 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-26T00:25:51.982004+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=51, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-26T00:24:23.468323+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=23, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-26T00:25:47.736821+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=47, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-26T00:24:23.468323+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=23, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-26T00:25:49.302264+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=49, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-26T00:25:49.772261+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=49, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-26T00:24:23.468323+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=23, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-26T00:25:50.799180+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=50, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-26T00:24:23.468323+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=23, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-26T00:25:53.106632+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=53, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.969 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-26T00:24:23.468323+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=23, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.970 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-26T00:24:23.468323+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=24, tm_sec=23, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.970 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-26T00:25:47.549742+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=25, tm_sec=47, tm_wday=4, tm_yday=117, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=26, tm_hour=0, tm_min=37, tm_sec=31, tm_wday=4, tm_yday=117, tm_isdst=0) 2024-04-26T00:37:33.970 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-26T00:37:53.970 DEBUG:teuthology.orchestra.run.smithi072:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-26T00:37:54.263 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-04-26T00:37:54.263 INFO:teuthology.orchestra.run.smithi072.stderr:dumped all 2024-04-26T00:37:54.275 INFO:teuthology.orchestra.run.smithi072.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-04-26T00:37:52.601102+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1453432,"kb_used_data":78432,"kb_used_omap":12,"kb_used_meta":1374963,"kb_avail":281662088,"statfs":{"total":289910292480,"available":288421978112,"internally_reserved":0,"allocated":80314368,"data_stored":79311961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13204,"internal_metadata":1407962220},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002685"},"pg_stats":[{"pgid":"2.f","version":"44'21260","reported_seq":28650,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:36.251643+0000","last_change":"2024-04-26T00:37:36.251671+0000","last_active":"2024-04-26T00:37:36.251643+0000","last_peered":"2024-04-26T00:37:36.251643+0000","last_clean":"2024-04-26T00:37:36.251643+0000","last_became_active":"2024-04-26T00:35:28.954632+0000","last_became_peered":"2024-04-26T00:35:28.954632+0000","last_unstale":"2024-04-26T00:37:36.251643+0000","last_undegraded":"2024-04-26T00:37:36.251643+0000","last_fullsized":"2024-04-26T00:37:36.251643+0000","mapping_epoch":47,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21260","last_scrub_stamp":"2024-04-26T00:37:36.251534+0000","last_deep_scrub":"44'21260","last_deep_scrub_stamp":"2024-04-26T00:37:36.251534+0000","last_clean_scrub_stamp":"2024-04-26T00:37:36.251534+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:26:23.381632+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21260,"num_write_kb":42520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3631,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"44'21682","reported_seq":21821,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:36.706773+0000","last_change":"2024-04-26T00:37:36.706801+0000","last_active":"2024-04-26T00:37:36.706773+0000","last_peered":"2024-04-26T00:37:36.706773+0000","last_clean":"2024-04-26T00:37:36.706773+0000","last_became_active":"2024-04-26T00:25:08.972964+0000","last_became_peered":"2024-04-26T00:25:08.972964+0000","last_unstale":"2024-04-26T00:37:36.706773+0000","last_undegraded":"2024-04-26T00:37:36.706773+0000","last_fullsized":"2024-04-26T00:37:36.706773+0000","mapping_epoch":28,"log_start":"38'11600","ondisk_log_start":"38'11600","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21682","last_scrub_stamp":"2024-04-26T00:37:36.706662+0000","last_deep_scrub":"44'21682","last_deep_scrub_stamp":"2024-04-26T00:37:36.706662+0000","last_clean_scrub_stamp":"2024-04-26T00:37:36.706662+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:39:31.558198+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18787,"num_write_kb":31796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"44'21284","reported_seq":28604,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:39.434878+0000","last_change":"2024-04-26T00:37:39.434922+0000","last_active":"2024-04-26T00:37:39.434878+0000","last_peered":"2024-04-26T00:37:39.434878+0000","last_clean":"2024-04-26T00:37:39.434878+0000","last_became_active":"2024-04-26T00:31:52.340235+0000","last_became_peered":"2024-04-26T00:31:52.340235+0000","last_unstale":"2024-04-26T00:37:39.434878+0000","last_undegraded":"2024-04-26T00:37:39.434878+0000","last_fullsized":"2024-04-26T00:37:39.434878+0000","mapping_epoch":41,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":42,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21284","last_scrub_stamp":"2024-04-26T00:37:39.434730+0000","last_deep_scrub":"44'21284","last_deep_scrub_stamp":"2024-04-26T00:37:39.434730+0000","last_clean_scrub_stamp":"2024-04-26T00:37:39.434730+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:55:48.944347+0000","scrub_duration":25,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18472,"num_write_kb":31325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":10080256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"44'21098","reported_seq":21193,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:33.767560+0000","last_change":"2024-04-26T00:37:33.767585+0000","last_active":"2024-04-26T00:37:33.767560+0000","last_peered":"2024-04-26T00:37:33.767560+0000","last_clean":"2024-04-26T00:37:33.767560+0000","last_became_active":"2024-04-26T00:25:08.968690+0000","last_became_peered":"2024-04-26T00:25:08.968690+0000","last_unstale":"2024-04-26T00:37:33.767560+0000","last_undegraded":"2024-04-26T00:37:33.767560+0000","last_fullsized":"2024-04-26T00:37:33.767560+0000","mapping_epoch":28,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21098","last_scrub_stamp":"2024-04-26T00:37:33.767456+0000","last_deep_scrub":"44'21098","last_deep_scrub_stamp":"2024-04-26T00:37:33.767456+0000","last_clean_scrub_stamp":"2024-04-26T00:37:33.767456+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:16:05.898132+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21098,"num_write_kb":42196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"44'21766","reported_seq":22039,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:34.213316+0000","last_change":"2024-04-26T00:37:34.213339+0000","last_active":"2024-04-26T00:37:34.213316+0000","last_peered":"2024-04-26T00:37:34.213316+0000","last_clean":"2024-04-26T00:37:34.213316+0000","last_became_active":"2024-04-26T00:25:08.973819+0000","last_became_peered":"2024-04-26T00:25:08.973819+0000","last_unstale":"2024-04-26T00:37:34.213316+0000","last_undegraded":"2024-04-26T00:37:34.213316+0000","last_fullsized":"2024-04-26T00:37:34.213316+0000","mapping_epoch":28,"log_start":"38'11700","ondisk_log_start":"38'11700","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21766","last_scrub_stamp":"2024-04-26T00:37:34.213230+0000","last_deep_scrub":"44'21766","last_deep_scrub_stamp":"2024-04-26T00:37:34.213230+0000","last_clean_scrub_stamp":"2024-04-26T00:37:34.213230+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:49:23.081772+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18167,"num_write_kb":29792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"44'21275","reported_seq":22083,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:45.481351+0000","last_change":"2024-04-26T00:37:45.481376+0000","last_active":"2024-04-26T00:37:45.481351+0000","last_peered":"2024-04-26T00:37:45.481351+0000","last_clean":"2024-04-26T00:37:45.481351+0000","last_became_active":"2024-04-26T00:37:25.338693+0000","last_became_peered":"2024-04-26T00:37:25.338693+0000","last_unstale":"2024-04-26T00:37:45.481351+0000","last_undegraded":"2024-04-26T00:37:45.481351+0000","last_fullsized":"2024-04-26T00:37:45.481351+0000","mapping_epoch":59,"log_start":"38'11200","ondisk_log_start":"38'11200","created":13,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21275","last_scrub_stamp":"2024-04-26T00:37:45.481251+0000","last_deep_scrub":"44'21275","last_deep_scrub_stamp":"2024-04-26T00:37:45.481251+0000","last_clean_scrub_stamp":"2024-04-26T00:37:45.481251+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:03:50.234803+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18450,"num_write_kb":31260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3575,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"44'20938","reported_seq":28352,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:38.412720+0000","last_change":"2024-04-26T00:37:38.412749+0000","last_active":"2024-04-26T00:37:38.412720+0000","last_peered":"2024-04-26T00:37:38.412720+0000","last_clean":"2024-04-26T00:37:38.412720+0000","last_became_active":"2024-04-26T00:29:56.884831+0000","last_became_peered":"2024-04-26T00:29:56.884831+0000","last_unstale":"2024-04-26T00:37:38.412720+0000","last_undegraded":"2024-04-26T00:37:38.412720+0000","last_fullsized":"2024-04-26T00:37:38.412720+0000","mapping_epoch":39,"log_start":"38'10900","ondisk_log_start":"38'10900","created":13,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'20938","last_scrub_stamp":"2024-04-26T00:37:38.412618+0000","last_deep_scrub":"44'20938","last_deep_scrub_stamp":"2024-04-26T00:37:38.412618+0000","last_clean_scrub_stamp":"2024-04-26T00:37:38.412618+0000","objects_scrubbed":0,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T00:40:18.686643+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18140,"num_write_kb":30696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3614,"num_bytes_recovered":14794752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"44'21406","reported_seq":21659,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:33.176890+0000","last_change":"2024-04-26T00:37:33.176915+0000","last_active":"2024-04-26T00:37:33.176890+0000","last_peered":"2024-04-26T00:37:33.176890+0000","last_clean":"2024-04-26T00:37:33.176890+0000","last_became_active":"2024-04-26T00:25:08.969243+0000","last_became_peered":"2024-04-26T00:25:08.969243+0000","last_unstale":"2024-04-26T00:37:33.176890+0000","last_undegraded":"2024-04-26T00:37:33.176890+0000","last_fullsized":"2024-04-26T00:37:33.176890+0000","mapping_epoch":28,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21406","last_scrub_stamp":"2024-04-26T00:37:33.176803+0000","last_deep_scrub":"44'21406","last_deep_scrub_stamp":"2024-04-26T00:37:33.176803+0000","last_clean_scrub_stamp":"2024-04-26T00:37:33.176803+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T08:03:17.468083+0000","scrub_duration":28,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18604,"num_write_kb":31608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:33.365965+0000","last_change":"2024-04-26T00:37:33.365984+0000","last_active":"2024-04-26T00:37:33.365965+0000","last_peered":"2024-04-26T00:37:33.365965+0000","last_clean":"2024-04-26T00:37:33.365965+0000","last_became_active":"2024-04-26T00:26:01.363961+0000","last_became_peered":"2024-04-26T00:26:01.363961+0000","last_unstale":"2024-04-26T00:37:33.365965+0000","last_undegraded":"2024-04-26T00:37:33.365965+0000","last_fullsized":"2024-04-26T00:37:33.365965+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:37:33.365897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:37:33.365897+0000","last_clean_scrub_stamp":"2024-04-26T00:37:33.365897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T01:46:59.775044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"44'21067","reported_seq":21682,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:41.430033+0000","last_change":"2024-04-26T00:37:41.430057+0000","last_active":"2024-04-26T00:37:41.430033+0000","last_peered":"2024-04-26T00:37:41.430033+0000","last_clean":"2024-04-26T00:37:41.430033+0000","last_became_active":"2024-04-26T00:35:50.009993+0000","last_became_peered":"2024-04-26T00:35:50.009993+0000","last_unstale":"2024-04-26T00:37:41.430033+0000","last_undegraded":"2024-04-26T00:37:41.430033+0000","last_fullsized":"2024-04-26T00:37:41.430033+0000","mapping_epoch":49,"log_start":"38'11000","ondisk_log_start":"38'11000","created":13,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21067","last_scrub_stamp":"2024-04-26T00:37:41.429933+0000","last_deep_scrub":"44'21067","last_deep_scrub_stamp":"2024-04-26T00:37:41.429933+0000","last_clean_scrub_stamp":"2024-04-26T00:37:41.429933+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:08:38.045587+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:34.726027+0000","last_change":"2024-04-26T00:37:34.726027+0000","last_active":"2024-04-26T00:37:34.726027+0000","last_peered":"2024-04-26T00:37:34.726027+0000","last_clean":"2024-04-26T00:37:34.726027+0000","last_became_active":"2024-04-26T00:25:08.969498+0000","last_became_peered":"2024-04-26T00:25:08.969498+0000","last_unstale":"2024-04-26T00:37:34.726027+0000","last_undegraded":"2024-04-26T00:37:34.726027+0000","last_fullsized":"2024-04-26T00:37:34.726027+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:37:34.725902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:37:34.725902+0000","last_clean_scrub_stamp":"2024-04-26T00:37:34.725902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:47:28.936154+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"44'21424","reported_seq":28853,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:37.298978+0000","last_change":"2024-04-26T00:37:37.299017+0000","last_active":"2024-04-26T00:37:37.298978+0000","last_peered":"2024-04-26T00:37:37.298978+0000","last_clean":"2024-04-26T00:37:37.298978+0000","last_became_active":"2024-04-26T00:36:08.083554+0000","last_became_peered":"2024-04-26T00:36:08.083554+0000","last_unstale":"2024-04-26T00:37:37.298978+0000","last_undegraded":"2024-04-26T00:37:37.298978+0000","last_fullsized":"2024-04-26T00:37:37.298978+0000","mapping_epoch":51,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21424","last_scrub_stamp":"2024-04-26T00:37:37.298874+0000","last_deep_scrub":"44'21424","last_deep_scrub_stamp":"2024-04-26T00:37:37.298874+0000","last_clean_scrub_stamp":"2024-04-26T00:37:37.298874+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:28:00.664678+0000","scrub_duration":31,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18205,"num_write_kb":29988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3624,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":124,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:35.208149+0000","last_change":"2024-04-26T00:37:35.208177+0000","last_active":"2024-04-26T00:37:35.208149+0000","last_peered":"2024-04-26T00:37:35.208149+0000","last_clean":"2024-04-26T00:37:35.208149+0000","last_became_active":"2024-04-26T00:26:01.454838+0000","last_became_peered":"2024-04-26T00:26:01.454838+0000","last_unstale":"2024-04-26T00:37:35.208149+0000","last_undegraded":"2024-04-26T00:37:35.208149+0000","last_fullsized":"2024-04-26T00:37:35.208149+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:37:35.208080+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:37:35.208080+0000","last_clean_scrub_stamp":"2024-04-26T00:37:35.208080+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:45:23.196881+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"44'21409","reported_seq":28932,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:38.774255+0000","last_change":"2024-04-26T00:37:38.774280+0000","last_active":"2024-04-26T00:37:38.774255+0000","last_peered":"2024-04-26T00:37:38.774255+0000","last_clean":"2024-04-26T00:37:38.774255+0000","last_became_active":"2024-04-26T00:35:08.919951+0000","last_became_peered":"2024-04-26T00:35:08.919951+0000","last_unstale":"2024-04-26T00:37:38.774255+0000","last_undegraded":"2024-04-26T00:37:38.774255+0000","last_fullsized":"2024-04-26T00:37:38.774255+0000","mapping_epoch":45,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21409","last_scrub_stamp":"2024-04-26T00:37:38.774139+0000","last_deep_scrub":"44'21409","last_deep_scrub_stamp":"2024-04-26T00:37:38.774139+0000","last_clean_scrub_stamp":"2024-04-26T00:37:38.774139+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T07:43:44.707980+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18409,"num_write_kb":30828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3683,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":117,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:35.403448+0000","last_change":"2024-04-26T00:37:35.403492+0000","last_active":"2024-04-26T00:37:35.403448+0000","last_peered":"2024-04-26T00:37:35.403448+0000","last_clean":"2024-04-26T00:37:35.403448+0000","last_became_active":"2024-04-26T00:26:01.433433+0000","last_became_peered":"2024-04-26T00:26:01.433433+0000","last_unstale":"2024-04-26T00:37:35.403448+0000","last_undegraded":"2024-04-26T00:37:35.403448+0000","last_fullsized":"2024-04-26T00:37:35.403448+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:37:35.403356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:37:35.403356+0000","last_clean_scrub_stamp":"2024-04-26T00:37:35.403356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:48:45.096219+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"44'21120","reported_seq":28366,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:37.730090+0000","last_change":"2024-04-26T00:37:37.730117+0000","last_active":"2024-04-26T00:37:37.730090+0000","last_peered":"2024-04-26T00:37:37.730090+0000","last_clean":"2024-04-26T00:37:37.730090+0000","last_became_active":"2024-04-26T00:27:38.085969+0000","last_became_peered":"2024-04-26T00:27:38.085969+0000","last_unstale":"2024-04-26T00:37:37.730090+0000","last_undegraded":"2024-04-26T00:37:37.730090+0000","last_fullsized":"2024-04-26T00:37:37.730090+0000","mapping_epoch":37,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21120","last_scrub_stamp":"2024-04-26T00:37:37.729954+0000","last_deep_scrub":"44'21120","last_deep_scrub_stamp":"2024-04-26T00:37:37.729954+0000","last_clean_scrub_stamp":"2024-04-26T00:37:37.729954+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T02:45:51.537452+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18125,"num_write_kb":30272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3545,"num_bytes_recovered":14512128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"44'21304","reported_seq":22059,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:44.488325+0000","last_change":"2024-04-26T00:37:44.488350+0000","last_active":"2024-04-26T00:37:44.488325+0000","last_peered":"2024-04-26T00:37:44.488325+0000","last_clean":"2024-04-26T00:37:44.488325+0000","last_became_active":"2024-04-26T00:37:05.257561+0000","last_became_peered":"2024-04-26T00:37:05.257561+0000","last_unstale":"2024-04-26T00:37:44.488325+0000","last_undegraded":"2024-04-26T00:37:44.488325+0000","last_fullsized":"2024-04-26T00:37:44.488325+0000","mapping_epoch":57,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21304","last_scrub_stamp":"2024-04-26T00:37:44.488235+0000","last_deep_scrub":"44'21304","last_deep_scrub_stamp":"2024-04-26T00:37:44.488235+0000","last_clean_scrub_stamp":"2024-04-26T00:37:44.488235+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T04:55:10.908383+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18225,"num_write_kb":30296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3636,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":113,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:36.413776+0000","last_change":"2024-04-26T00:37:36.413798+0000","last_active":"2024-04-26T00:37:36.413776+0000","last_peered":"2024-04-26T00:37:36.413776+0000","last_clean":"2024-04-26T00:37:36.413776+0000","last_became_active":"2024-04-26T00:26:01.485534+0000","last_became_peered":"2024-04-26T00:26:01.485534+0000","last_unstale":"2024-04-26T00:37:36.413776+0000","last_undegraded":"2024-04-26T00:37:36.413776+0000","last_fullsized":"2024-04-26T00:37:36.413776+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:37:36.413541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:37:36.413541+0000","last_clean_scrub_stamp":"2024-04-26T00:37:36.413541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:39:58.175230+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"44'21170","reported_seq":21840,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:43.465230+0000","last_change":"2024-04-26T00:37:43.465275+0000","last_active":"2024-04-26T00:37:43.465230+0000","last_peered":"2024-04-26T00:37:43.465230+0000","last_clean":"2024-04-26T00:37:43.465230+0000","last_became_active":"2024-04-26T00:36:46.209471+0000","last_became_peered":"2024-04-26T00:36:46.209471+0000","last_unstale":"2024-04-26T00:37:43.465230+0000","last_undegraded":"2024-04-26T00:37:43.465230+0000","last_fullsized":"2024-04-26T00:37:43.465230+0000","mapping_epoch":55,"log_start":"38'11100","ondisk_log_start":"38'11100","created":13,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21170","last_scrub_stamp":"2024-04-26T00:37:43.465120+0000","last_deep_scrub":"44'21170","last_deep_scrub_stamp":"2024-04-26T00:37:43.465120+0000","last_clean_scrub_stamp":"2024-04-26T00:37:43.465120+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T03:44:43.682641+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18052,"num_write_kb":29884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:35.683751+0000","last_change":"2024-04-26T00:37:35.683751+0000","last_active":"2024-04-26T00:37:35.683751+0000","last_peered":"2024-04-26T00:37:35.683751+0000","last_clean":"2024-04-26T00:37:35.683751+0000","last_became_active":"2024-04-26T00:25:08.975158+0000","last_became_peered":"2024-04-26T00:25:08.975158+0000","last_unstale":"2024-04-26T00:37:35.683751+0000","last_undegraded":"2024-04-26T00:37:35.683751+0000","last_fullsized":"2024-04-26T00:37:35.683751+0000","mapping_epoch":28,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":29,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:37:35.683365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:37:35.683365+0000","last_clean_scrub_stamp":"2024-04-26T00:37:35.683365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T06:45:51.564512+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"44'21420","reported_seq":22172,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:42.439366+0000","last_change":"2024-04-26T00:37:42.439391+0000","last_active":"2024-04-26T00:37:42.439366+0000","last_peered":"2024-04-26T00:37:42.439366+0000","last_clean":"2024-04-26T00:37:42.439366+0000","last_became_active":"2024-04-26T00:36:26.143987+0000","last_became_peered":"2024-04-26T00:36:26.143987+0000","last_unstale":"2024-04-26T00:37:42.439366+0000","last_undegraded":"2024-04-26T00:37:42.439366+0000","last_fullsized":"2024-04-26T00:37:42.439366+0000","mapping_epoch":53,"log_start":"38'11400","ondisk_log_start":"38'11400","created":13,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"44'21420","last_scrub_stamp":"2024-04-26T00:37:42.439266+0000","last_deep_scrub":"44'21420","last_deep_scrub_stamp":"2024-04-26T00:37:42.439266+0000","last_clean_scrub_stamp":"2024-04-26T00:37:42.439266+0000","objects_scrubbed":0,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:26:25.557743+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18116,"num_write_kb":29632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3657,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":113,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:37.399581+0000","last_change":"2024-04-26T00:37:37.399607+0000","last_active":"2024-04-26T00:37:37.399581+0000","last_peered":"2024-04-26T00:37:37.399581+0000","last_clean":"2024-04-26T00:37:37.399581+0000","last_became_active":"2024-04-26T00:26:01.486331+0000","last_became_peered":"2024-04-26T00:26:01.486331+0000","last_unstale":"2024-04-26T00:37:37.399581+0000","last_undegraded":"2024-04-26T00:37:37.399581+0000","last_fullsized":"2024-04-26T00:37:37.399581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:37:37.399470+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:37:37.399470+0000","last_clean_scrub_stamp":"2024-04-26T00:37:37.399470+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T05:09:27.960213+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:34.438743+0000","last_change":"2024-04-26T00:37:34.438763+0000","last_active":"2024-04-26T00:37:34.438743+0000","last_peered":"2024-04-26T00:37:34.438743+0000","last_clean":"2024-04-26T00:37:34.438743+0000","last_became_active":"2024-04-26T00:26:01.370118+0000","last_became_peered":"2024-04-26T00:26:01.370118+0000","last_unstale":"2024-04-26T00:37:34.438743+0000","last_undegraded":"2024-04-26T00:37:34.438743+0000","last_fullsized":"2024-04-26T00:37:34.438743+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T00:37:34.438659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T00:37:34.438659+0000","last_clean_scrub_stamp":"2024-04-26T00:37:34.438659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T12:31:12.354632+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"60'21323","reported_seq":28781,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-26T00:37:40.427140+0000","last_change":"2024-04-26T00:37:40.427179+0000","last_active":"2024-04-26T00:37:40.427140+0000","last_peered":"2024-04-26T00:37:40.427140+0000","last_clean":"2024-04-26T00:37:40.427140+0000","last_became_active":"2024-04-26T00:33:56.415379+0000","last_became_peered":"2024-04-26T00:33:56.415379+0000","last_unstale":"2024-04-26T00:37:40.427140+0000","last_undegraded":"2024-04-26T00:37:40.427140+0000","last_fullsized":"2024-04-26T00:37:40.427140+0000","mapping_epoch":43,"log_start":"38'11300","ondisk_log_start":"38'11300","created":13,"last_epoch_clean":44,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'21323","last_scrub_stamp":"2024-04-26T00:37:40.427059+0000","last_deep_scrub":"60'21323","last_deep_scrub_stamp":"2024-04-26T00:37:40.427059+0000","last_clean_scrub_stamp":"2024-04-26T00:37:40.427059+0000","objects_scrubbed":1,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T09:09:03.337272+0000","scrub_duration":41,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18372,"num_write_kb":30868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3635,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":483,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":280482,"num_write_kb":482961,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39744,"num_bytes_recovered":39387136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":78815232,"data_stored":78808006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160746,"ondisk_log_size":160746,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":1,"up_from":34,"seq":146028888208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":527796,"kb_used_data":492,"kb_used_omap":4,"kb_used_meta":527291,"kb_avail":93844044,"statfs":{"total":96636764160,"available":96096301056,"internally_reserved":0,"allocated":503808,"data_stored":168468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4401,"internal_metadata":539946703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":22,"seq":94489280675,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":538980,"kb_used_data":43676,"kb_used_omap":4,"kb_used_meta":495291,"kb_avail":93832860,"statfs":{"total":96636764160,"available":96084848640,"internally_reserved":0,"allocated":44724224,"data_stored":44388884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":507178701},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":28,"seq":120259084447,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":386656,"kb_used_data":34264,"kb_used_omap":4,"kb_used_meta":352379,"kb_avail":93985184,"statfs":{"total":96636764160,"available":96240828416,"internally_reserved":0,"allocated":35086336,"data_stored":34754609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4400,"internal_metadata":360836816},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":44224512,"data_stored":44220899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":34586624,"data_stored":34586624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T00:37:54.277 DEBUG:teuthology.orchestra.run.smithi072:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-26T00:37:54.582 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-26T00:37:54.583 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-26T00:37:54.583 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-26T00:37:54.583 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:37:59.437 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-04-26T00:38:00.684 INFO:tasks.ceph.osd.0:Stopped 2024-04-26T00:38:00.685 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-26T00:38:00.685 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:38:00.765 INFO:tasks.ceph.osd.1:Stopped 2024-04-26T00:38:00.765 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-26T00:38:00.765 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:38:04.840 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-04-26T00:38:04.840 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-04-26T00:38:06.867 INFO:tasks.ceph.osd.2:Stopped 2024-04-26T00:38:06.867 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-26T00:38:06.868 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-26T00:38:06.868 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:38:06.897 INFO:tasks.ceph.mgr.x:Stopped 2024-04-26T00:38:06.898 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-26T00:38:06.898 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-26T00:38:06.898 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:38:06.952 INFO:tasks.ceph.mon.a:Stopped 2024-04-26T00:38:06.952 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-26T00:38:06.952 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:38:07.005 INFO:tasks.ceph.mon.b:Stopped 2024-04-26T00:38:07.005 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-26T00:38:07.005 INFO:teuthology.orchestra.run:waiting for 300 2024-04-26T00:38:07.058 INFO:tasks.ceph.mon.c:Stopped 2024-04-26T00:38:07.058 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-26T00:38:07.058 DEBUG:teuthology.orchestra.run.smithi072:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v '\(MON_DOWN\)' | head -n 1 2024-04-26T00:38:07.166 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:38:07.166 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-26T00:38:08.371 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:38:08.372 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-26T00:38:08.511 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:38:08.511 DEBUG:teuthology.orchestra.run.smithi072:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-26T00:38:08.579 INFO:tasks.ceph:Archiving mon data... 2024-04-26T00:38:08.580 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897/data/mon.a.tgz 2024-04-26T00:38:08.580 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2024-04-26T00:38:08.609 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.2qgpQEkl4q 2024-04-26T00:38:08.610 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.2qgpQEkl4q 2024-04-26T00:38:08.776 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.2qgpQEkl4q 2024-04-26T00:38:08.875 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.2qgpQEkl4q is 272KB 2024-04-26T00:38:08.969 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.2qgpQEkl4q 2024-04-26T00:38:08.983 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897/data/mon.b.tgz 2024-04-26T00:38:08.983 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2024-04-26T00:38:09.037 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.wqUvhH8NF3 2024-04-26T00:38:09.038 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.wqUvhH8NF3 2024-04-26T00:38:09.148 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.wqUvhH8NF3 2024-04-26T00:38:09.230 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.wqUvhH8NF3 is 275KB 2024-04-26T00:38:09.324 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.wqUvhH8NF3 2024-04-26T00:38:09.339 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897/data/mon.c.tgz 2024-04-26T00:38:09.339 DEBUG:teuthology.orchestra.run.smithi072:> mktemp 2024-04-26T00:38:09.392 INFO:teuthology.orchestra.run.smithi072.stdout:/tmp/tmp.RNczsnkvJP 2024-04-26T00:38:09.393 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.RNczsnkvJP 2024-04-26T00:38:09.503 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 0666 /tmp/tmp.RNczsnkvJP 2024-04-26T00:38:09.584 DEBUG:teuthology.orchestra.remote:smithi072:/tmp/tmp.RNczsnkvJP is 275KB 2024-04-26T00:38:09.680 DEBUG:teuthology.orchestra.run.smithi072:> rm -fr /tmp/tmp.RNczsnkvJP 2024-04-26T00:38:09.695 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-26T00:38:09.695 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-26T00:38:09.794 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-26T00:38:09.794 INFO:tasks.ceph:Archiving crash dumps... 2024-04-26T00:38:09.796 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897/remote/smithi072/crash 2024-04-26T00:38:09.797 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-26T00:38:09.841 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-04-26T00:38:09.857 INFO:tasks.ceph:Compressing logs... 2024-04-26T00:38:09.857 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T00:38:10.062 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34217.log 2024-04-26T00:38:10.062 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36645.log 2024-04-26T00:38:10.062 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36006.log 2024-04-26T00:38:10.062 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34217.log.gz 2024-04-26T00:38:10.062 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36645.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34175.log 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36645.log.gz 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33755.log 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36006.log.gz 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34175.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.34175.log.gz 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr: /var/log/ceph/ceph-client.admin.33794.log 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36228.log 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33755.log.gz 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47180.log 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33794.log.gz 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40528.log 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36228.log.gz 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50288.log 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47180.log.gz 2024-04-26T00:38:10.063 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46339.log 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40528.log.gz 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50288.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34766.log 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50288.log.gz 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49196.log 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46339.log.gz 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40883.log 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34766.log.gz 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47432.log 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49196.log.gz 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34298.log 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40883.log.gz 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47306.log 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47432.log.gz 2024-04-26T00:38:10.064 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46255.log 2024-04-26T00:38:10.065 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34298.log.gz 2024-04-26T00:38:10.065 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47138.log 2024-04-26T00:38:10.065 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47306.log.gz 2024-04-26T00:38:10.065 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42437.log 2024-04-26T00:38:10.065 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46255.log.gz 2024-04-26T00:38:10.065 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47516.log 2024-04-26T00:38:10.065 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47138.log.gz 2024-04-26T00:38:10.065 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49070.log 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42437.log.gz 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40341.log 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47516.log.gz 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40799.log 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49070.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49070.log.gz 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42521.log 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40341.log.gz 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42988.log 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40799.log.gz 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34550.log 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42521.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42521.log.gz 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46087.log 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42988.log.gz 2024-04-26T00:38:10.066 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43395.log 2024-04-26T00:38:10.067 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34550.log.gz 2024-04-26T00:38:10.067 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37274.log/var/log/ceph/ceph-client.admin.46087.log: 2024-04-26T00:38:10.067 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46087.log.gz 2024-04-26T00:38:10.069 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36682.log 2024-04-26T00:38:10.069 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43395.log.gz 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36904.log 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37274.log.gz 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46633.log 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36682.log.gz 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41555.log 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36904.log.gz 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45373.log 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46633.log.gz 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41387.log 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41555.log.gz 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34808.log 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45373.log.gz 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44471.log 2024-04-26T00:38:10.070 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41387.log.gz 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50750.log 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34808.log.gz 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35636.log 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44471.log.gz 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50498.log 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50750.log.gz 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42766.log 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35636.log.gz 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42803.log 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50498.log.gz 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37459.log 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42766.log.gz 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40109.log 2024-04-26T00:38:10.071 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42803.log.gz 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45583.log 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37459.log.gz 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39837.log 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40109.log.gz 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49658.log 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45583.log.gz 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50120.log 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39837.log.gz 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34049.log 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49658.log.gz 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46003.log 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50120.log.gz 2024-04-26T00:38:10.072 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48944.log 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34049.log.gz 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35858.log 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46003.log.gz 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36867.log 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48944.log.gz 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35858.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45415.log 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35858.log.gz 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44659.log 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36867.log.gz 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49616.log 2024-04-26T00:38:10.073 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45415.log.gz 2024-04-26T00:38:10.074 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36497.log 2024-04-26T00:38:10.074 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-26T00:38:10.074 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39940.log 2024-04-26T00:38:10.074 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42729.log 2024-04-26T00:38:10.074 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46591.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35307.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35525.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35032.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48776.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30731.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43562.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37422.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42951.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43358.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49154.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41345.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49490.log 2024-04-26T00:38:10.075 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36432.log 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48650.log 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48398.log 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44659.log: /var/log/ceph/ceph-client.admin.49616.log: /var/log/ceph/ceph-client.admin.36497.log: /var/log/ceph/ceph-client.admin.48650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44659.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45331.log 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36497.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49616.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39940.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48650.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39940.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46591.log: /var/log/ceph/ceph-client.admin.35307.log: /var/log/ceph/ceph-client.admin.48398.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.42729.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr: /var/log/ceph/ceph-client.admin.40677.log 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46591.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35307.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35525.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35032.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48398.log.gz 2024-04-26T00:38:10.076 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35032.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48776.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50078.log 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45331.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.30731.log: /var/log/ceph/ceph-client.admin.43562.log: /var/log/ceph/ceph-osd.2.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30731.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43562.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40677.log: /var/log/ceph/ceph-client.admin.37422.log: /var/log/ceph/ceph-client.admin.42951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40677.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5/var/log/ceph/ceph-client.admin.43358.log: --verbose -- /var/log/ceph/ceph-client.admin.40444.log 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37422.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42951.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43358.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49154.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41345.log: /var/log/ceph/ceph-client.admin.49490.log: /var/log/ceph/ceph-client.admin.50078.log: /var/log/ceph/ceph-client.admin.36432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41345.log.gz 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-04-26T00:38:10.077 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49490.log.gz 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50078.log.gz 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36432.log.gz 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40700.log 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40444.log.gz 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38466.log 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44953.log 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38466.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38466.log.gz 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37385.log 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44953.log.gz 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35414.log 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47642.log 2024-04-26T00:38:10.078 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37385.log.gz 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35117.log 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35414.log.gz 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47642.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34679.log 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47642.log.gz 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45877.log 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35117.log.gz 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36302.log 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34679.log.gz 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45667.log 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45877.log.gz 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50624.log 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36302.log.gz 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38612.log 2024-04-26T00:38:10.079 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45667.log.gz 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45163.log 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50624.log.gz 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34424.log 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38612.log.gz 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr: 95.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45163.log.gz 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35784.log 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34424.log.gz 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34299.log 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36043.log 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35784.log.gz 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49448.log 2024-04-26T00:38:10.080 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34299.log.gz 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36043.log.gz 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47684.log 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49448.log.gz 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40173.log 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35265.log 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47684.log.gz 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38696.log 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40173.log.gz 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36534.log 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35265.log.gz 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45835.log 2024-04-26T00:38:10.081 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38696.log.gz 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36608.log 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36534.log.gz 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38528.log 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45835.log.gz 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36608.log.gzgzip 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46801.log 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46171.log 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38528.log.gz 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40299.log 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46801.log.gz 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44701.log 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46171.log.gz 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48020.log 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40299.log.gz 2024-04-26T00:38:10.082 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44701.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37163.log 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44701.log.gz 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50708.log 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48020.log.gz 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33667.log 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37163.log.gz 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41471.log 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50708.log.gz 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36080.log 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33667.log.gz 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33419.log 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41471.log.gz 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37015.log 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36080.log.gz 2024-04-26T00:38:10.083 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45121.log 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33419.log.gz 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47768.log 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37015.log.gz 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48860.log 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45121.log.gz 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47390.log 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47768.log.gz 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34297.log 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48860.log.gz 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50372.log 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47390.log.gz 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49868.log 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34297.log.gz 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42143.log 2024-04-26T00:38:10.084 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50372.log.gz 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49280.log 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49868.log.gz 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38676.log 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42143.log.gz 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39941.log 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49280.log.gz 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38676.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.38676.log.gz -- 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr: /var/log/ceph/ceph-client.admin.50456.log 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33503.log 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr: 95.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39941.log.gz 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43284.log 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50456.log.gz 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44513.log 2024-04-26T00:38:10.085 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33503.log.gz 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43321.log 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43284.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43284.log.gz 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48986.log 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44513.log.gz 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43321.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46549.log 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43321.log.gz 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48986.log.gz 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45247.log 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46549.log.gz 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35895.log 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41765.log 2024-04-26T00:38:10.086 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45247.log.gz 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33881.log 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35895.log.gz 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5/var/log/ceph/ceph-client.admin.41765.log: --verbose -- /var/log/ceph/ceph-client.admin.36191.log 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41765.log.gz 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38424.log 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33881.log.gz 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36941.log 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36191.log.gz 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41009.log 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38424.log.gz 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36941.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36941.log.gz 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41009.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.41009.log.gz -- 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr: /var/log/ceph/ceph-client.admin.36793.log 2024-04-26T00:38:10.087 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42692.log 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42163.log 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36793.log.gz 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42077.log 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42692.log.gz 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50036.log 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42163.log.gz 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50582.log 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42077.log.gz 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41051.log 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50036.log.gz 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37052.log 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50582.log.gz 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43506.log 2024-04-26T00:38:10.088 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41051.log.gz 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46507.log 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37052.log.gz 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35451.log 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43506.log.gz 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34091.log 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46507.log.gz 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42914.log 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35451.log.gz 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41681.log 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34091.log.gz 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43136.log 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42914.log.gz 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41681.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.41681.log.gz --verbose 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr: -- /var/log/ceph/ceph-client.admin.43062.log 2024-04-26T00:38:10.089 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35969.log 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43136.log.gz 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46927.log 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43062.log.gz 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42227.log 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35969.log.gz 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47011.log 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46927.log.gz 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49994.log 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42227.log.gz 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49532.log 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47011.log.gz 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48356.log 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49994.log.gz 2024-04-26T00:38:10.090 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50666.log 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49532.log.gz 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48356.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48356.log.gz 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49910.log 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50666.log.gz 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46213.log 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48104.log 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49910.log.gz 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41891.log 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46213.log.gz 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48104.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48104.log.gz 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41891.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38886.log 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41891.log.gz 2024-04-26T00:38:10.091 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42017.log 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41429.log 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38886.log.gz 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr:gzip/var/log/ceph/ceph-client.admin.42017.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38527.log 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42017.log.gz 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45205.log 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41429.log.gz 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36830.log 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38527.log.gz 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42353.log 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45205.log.gz 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33545.log 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36830.log.gz 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40024.log 2024-04-26T00:38:10.092 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42353.log.gz 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30632.log 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33545.log.gz 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36376.log 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40024.log.gz 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.30632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34551.log 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30632.log.gz 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33669.log 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36376.log.gz 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35488.log 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34551.log.gz 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33669.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46297.log 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33669.log.gz 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40967.log 2024-04-26T00:38:10.093 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35488.log.gz 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46297.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46297.log.gz 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46843.log 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40967.log.gz 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38760.log 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46843.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36756.log 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46843.log.gz 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48566.log 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38760.log.gz 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42840.log 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36756.log.gz 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40612.log 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48566.log.gz 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49406.log 2024-04-26T00:38:10.094 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42840.log.gz 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30804.log 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40612.log.gz 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41513.log 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49406.log.gz 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.30804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40445.log 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30804.log.gz 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41513.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34948.log 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41513.log.gz 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38802.log 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40445.log.gz 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43173.log 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34948.log.gz 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35747.log 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38802.log.gz 2024-04-26T00:38:10.095 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49742.log 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43173.log.gz 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49952.log 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35747.log.gz 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49742.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47558.log 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49742.log.gz 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50204.log 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49952.log.gz 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40108.log 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47558.log.gz 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50204.log.gz 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40841.log 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40025.log 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40108.log.gz 2024-04-26T00:38:10.096 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40841.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50330.log 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40841.log.gz 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45919.log 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40025.log.gz 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38382.log 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50330.log.gz 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44827.log 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45919.log.gz 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41807.log 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38382.log.gz 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48188.log 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44827.log.gz 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42877.log 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41807.log.gz 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48188.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41933.log 2024-04-26T00:38:10.097 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48188.log.gz 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46129.log 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42877.log.gz 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41933.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25835.log 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41933.log.gz 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37200.log 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46129.log.gz 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph.tmp-client.admin.25835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44911.log 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25835.log.gz 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35201.log 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37200.log.gz 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45625.log 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44911.log.gz 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34133.log 2024-04-26T00:38:10.098 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35201.log.gz 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35033.log 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45625.log.gz 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41849.log 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34133.log.gz 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35033.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48314.log 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35033.log.gz 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30552.log 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41849.log.gz 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48314.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50162.log 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48314.log.gz 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45079.log 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.30552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30552.log.gz 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33936.log 2024-04-26T00:38:10.099 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50162.log.gz 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47096.log 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45079.log.gz 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50790.log 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33936.log.gz 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47096.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47096.log.gz 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44555.log 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49784.log 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50790.log.gz 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44555.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45793.log 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44555.log.gz 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49784.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41639.log 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49784.log.gz 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46045.log 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45793.log.gz 2024-04-26T00:38:10.100 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47894.log 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41639.log.gz 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42618.log 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46045.log.gz 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44785.log 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47894.log.gz 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47810.log 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42618.log.gz 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33668.log 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44785.log.gz 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34007.log 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47810.log.gz 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43025.log 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33668.log.gz 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46969.log 2024-04-26T00:38:10.101 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34007.log.gz 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43469.log 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43025.log.gz 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47600.log 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46969.log.gz 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47264.log 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43469.log.gz 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48902.log 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47600.log.gz 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41303.log 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47264.log.gz 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34425.log 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48902.log.gz 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50414.log 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41303.log.gz 2024-04-26T00:38:10.102 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49700.log 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34425.log.gz 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40529.log 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50414.log.gz 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42395.log 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49700.log.gz 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46465.log 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40529.log.gz 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33363.log 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42395.log.gz 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46465.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46465.log.gzgzip 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.35673.log 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45709.log 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33363.log.gz 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36978.log 2024-04-26T00:38:10.103 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35673.log.gz 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45709.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48692.log 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45709.log.gz 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41723.log 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36978.log.gz 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35391.log 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48692.log.gz 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41135.log 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41723.log.gz 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48734.log 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35391.log.gz 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33461.log 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41135.log.gz 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36265.log 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48734.log.gz 2024-04-26T00:38:10.104 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36474.log 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33461.log.gz 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40257.log 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36265.log.gz 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35710.log 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36474.log.gz 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36719.log 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40257.log.gz 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35710.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41261.log 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35710.log.gz 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38928.log 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36719.log.gz 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46423.log 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41261.log.gz 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40613.log 2024-04-26T00:38:10.105 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38928.log.gz 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45037.log 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46423.log.gz 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45457.log 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40613.log.gz 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43099.log 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45037.log.gz 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33901.log 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45457.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45457.log.gz 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49112.log 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43099.log.gz 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49574.log 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33901.log.gz 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35116.log 2024-04-26T00:38:10.106 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49112.log.gz 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45751.log 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49574.log.gz 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40925.log 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35116.log.gz 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40757.log 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45751.log.gz 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37311.log 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40925.log.gz 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42479.log 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40757.log.gz 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37126.log 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37311.log.gz 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46675.log 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42479.log.gz 2024-04-26T00:38:10.107 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40383.log 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37126.log.gz 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35932.log 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46675.log.gz 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.40383.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42581.log 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40383.log.gz 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30433.log 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35932.log.gz 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50246.log 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42581.log.gz 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.30433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30592.log 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30433.log.gz 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36571.log 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50246.log.gz 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43247.log 2024-04-26T00:38:10.108 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.30592.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30592.log.gz 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37348.log 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36571.log.gz 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48146.log 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43247.log.gz 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41177.log 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37348.log.gz 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48818.log 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48146.log.gz 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33795.log 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41177.log.gz 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35200.log 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48818.log.gz 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33795.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44995.log 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33795.log.gz 2024-04-26T00:38:10.109 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41597.log 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35200.log.gz 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44995.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47978.log 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44995.log.gz 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49322.log 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41597.log.gz 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42544.log 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47978.log.gz 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49028.log 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49322.log.gz 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46717.log 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42544.log.gz 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43432.log 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49028.log.gz 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48482.log 2024-04-26T00:38:10.110 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46717.log.gz 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34511.log 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43432.log.gz 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47852.log 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48482.log.gz 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49238.log 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34511.log.gz 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48272.log 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47852.log.gz 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42247.log 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49238.log.gz 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48230.log 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48272.log.gz 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40215.log 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42247.log.gz 2024-04-26T00:38:10.111 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47474.log 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48230.log.gz 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.40215.log: /var/log/ceph/ceph-client.admin.45961.log 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40215.log.gz 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47348.log 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47474.log.gz 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45541.log 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45961.log.gz 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47348.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41975.log 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47348.log.gz 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48608.log 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45541.log.gz 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41093.log 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41975.log.gz 2024-04-26T00:38:10.112 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48524.log 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48608.log.gz 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48440.log 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41093.log.gz 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48524.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36154.log 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48524.log.gz 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42079.log 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48440.log.gz 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47936.log 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36154.log.gz 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47222.log 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42079.log.gz 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34949.log 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47936.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47936.log.gz 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44597.log 2024-04-26T00:38:10.113 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47222.log.gz 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34873.log 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34949.log.gz 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44597.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34637.log 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44597.log.gz 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39879.log 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34873.log.gz 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34637.log.gz 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46759.log 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38611.log 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.39879.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39879.log.gz 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35349.log 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46759.log.gz 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38611.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-26T00:38:10.114 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38611.log.gz 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34385.log 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35349.log.gz 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44869.log 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36339.log 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34385.log.gz 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45289.log 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44869.log.gz 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35821.log 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36339.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36339.log.gz 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48062.log 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45289.log.gz 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36117.log 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35821.log.gz 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.48062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43210.log 2024-04-26T00:38:10.115 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48062.log.gz 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.36117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30257.log 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36117.log.gz 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.43210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47726.log 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43210.log.gz 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.30257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33587.log 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30257.log.gz 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42655.log 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47726.log.gz 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.33587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50540.log 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33587.log.gz 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.42655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44743.log 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42655.log.gz 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.50540.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35562.log 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50540.log.gz 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.44743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41219.log 2024-04-26T00:38:10.116 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44743.log.gz 2024-04-26T00:38:10.117 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34850.log 2024-04-26T00:38:10.117 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35562.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35562.log.gz 2024-04-26T00:38:10.117 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.41219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37089.log 2024-04-26T00:38:10.117 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41219.log.gz 2024-04-26T00:38:10.117 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49826.log 2024-04-26T00:38:10.117 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34850.log.gz 2024-04-26T00:38:10.117 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47054.log 2024-04-26T00:38:10.117 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37089.log.gz 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.49826.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49826.log.gz 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45499.log 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.47054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38844.log 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47054.log.gz 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.45499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37237.log 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45499.log.gz 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.38844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35599.log 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38844.log.gz 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.37237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34721.log 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37237.log.gz 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.35599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46885.log 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35599.log.gz 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.34721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34721.log.gz 2024-04-26T00:38:10.118 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ceph-client.admin.46885.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46885.log.gz 2024-04-26T00:38:10.439 INFO:teuthology.orchestra.run.smithi072.stderr: 96.7% -- replaced with /var/log/ceph/ceph-client.admin.40700.log.gz 2024-04-26T00:38:10.511 INFO:teuthology.orchestra.run.smithi072.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-26T00:38:11.916 INFO:teuthology.orchestra.run.smithi072.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-26T00:38:12.412 INFO:teuthology.orchestra.run.smithi072.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-26T00:38:12.839 INFO:teuthology.orchestra.run.smithi072.stderr: 89.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-26T00:40:57.798 INFO:teuthology.orchestra.run.smithi072.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-26T00:44:20.699 INFO:teuthology.orchestra.run.smithi072.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-26T00:45:24.260 INFO:teuthology.orchestra.run.smithi072.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-26T00:45:24.261 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-04-26T00:45:24.261 INFO:teuthology.orchestra.run.smithi072.stderr:real 7m14.346s 2024-04-26T00:45:24.261 INFO:teuthology.orchestra.run.smithi072.stderr:user 5m14.306s 2024-04-26T00:45:24.261 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m12.016s 2024-04-26T00:45:24.262 INFO:tasks.ceph:Archiving logs... 2024-04-26T00:45:24.262 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897/remote/smithi072/log 2024-04-26T00:45:24.263 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T00:46:22.860 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T00:46:22.872 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T00:46:22.872 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T00:46:22.940 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T00:46:22.948 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T00:46:22.948 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T00:46:22.971 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-04-26T00:46:23.020 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T00:46:23.020 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-04-26T00:46:23.020 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv01.front.sepia.ceph.com 3 6 377 28 +1412us[+1420us] +/- 68ms 2024-04-26T00:46:23.020 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 27 -721us[ -721us] +/- 61ms 2024-04-26T00:46:23.020 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -547us[ -547us] +/- 65ms 2024-04-26T00:46:23.020 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T00:46:23.021 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T00:46:23.030 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T00:46:23.030 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T00:46:23.038 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T00:46:23.154 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T00:46:23.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:46:23.182 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T00:46:23.191 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T00:46:23.200 INFO:teuthology.task.internal:Duration was 1733.445832 seconds 2024-04-26T00:46:23.200 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T00:46:23.208 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T00:46:23.208 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T00:46:23.306 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T00:46:23.530 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T00:46:23.530 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2024-04-26T00:46:23.530 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T00:46:23.549 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T00:46:23.550 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T00:46:23.614 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T00:46:23.614 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T00:46:23.767 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T00:46:23.778 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T00:46:23.778 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T00:46:23.827 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T00:46:23.852 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T00:46:23.879 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2024-04-26T00:46:23.925 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T00:46:23.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T00:46:23.980 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T00:46:23.990 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T00:46:23.991 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-25_21:00:03-rados-squid-distro-default-smithi/7673897/remote/smithi072 2024-04-26T00:46:23.991 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T00:46:24.056 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T00:46:24.057 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T00:46:24.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T00:46:24.118 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T00:46:24.118 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T00:46:24.130 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T00:46:24.130 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T00:46:24.161 INFO:teuthology.orchestra.run.smithi072.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 00:46 /home/ubuntu/cephtest 2024-04-26T00:46:24.162 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T00:46:24.172 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T00:46:24.194 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_latest}} duration: 1733.4458320140839 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-26T00:46:24.194 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T00:46:24.270 INFO:teuthology.run:pass